html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue https://github.com/pydata/xarray/issues/1068#issuecomment-984940677,https://api.github.com/repos/pydata/xarray/issues/1068,984940677,IC_kwDOAMm_X846tQCF,1197350,2021-12-02T19:36:12Z,2021-12-02T19:36:12Z,MEMBER,"One solution to this problem might be the creation of a [custom Xarray backend](http://xarray.pydata.org/en/stable/internals/how-to-add-new-backend.html) for NASA EarthData. This backend could manage authentication with EDL and have its own documentation. If this package were maintained by NASA, it would close the feedback loop more effectively.","{""total_count"": 5, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 4, ""eyes"": 1}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-984920867,https://api.github.com/repos/pydata/xarray/issues/1068,984920867,IC_kwDOAMm_X846tLMj,1197350,2021-12-02T19:08:54Z,2021-12-02T19:08:54Z,MEMBER,"Just wanted to say how much I appreciate @betolink acting as a communication channel between Xarray and NASA. Users often end up on our issue tracker because Xarray raises errors whenever it can't read data. But the source of these problems is not with Xarray, it's with the upstream data provider. This also happens all the time with xmitgcm, e.g. https://github.com/MITgcm/xmitgcm/issues/266 It would be great if NASA had a better way to respond to these issues which didn't require that you ""know a guy"".","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-968993065,https://api.github.com/repos/pydata/xarray/issues/1068,968993065,IC_kwDOAMm_X845wakp,1197350,2021-11-15T14:58:05Z,2021-11-15T14:58:05Z,MEMBER,At what point do we escalate this issue to NASA? Is there a channel via which they can receive and respond to user feedback?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-966324523,https://api.github.com/repos/pydata/xarray/issues/1068,966324523,IC_kwDOAMm_X845mPEr,1197350,2021-11-11T13:59:55Z,2021-11-11T13:59:55Z,MEMBER,"I'd like to tag @betolink in this issue. He knows quite a bit about both Xarray and Earthdata login. Maybe he can help us get to the bottom of these problems. Luis, any ideas?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-624226394,https://api.github.com/repos/pydata/xarray/issues/1068,624226394,MDEyOklzc3VlQ29tbWVudDYyNDIyNjM5NA==,1197350,2020-05-05T18:23:57Z,2020-05-05T18:23:57Z,MEMBER,"> So, I tried Ryan's example, and got to the same error, where do you accept the EULA? It doesn't pop up on screen. https://urs.earthdata.nasa.gov/app_eula/nasa_gesdisc_data_archive","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-582432731,https://api.github.com/repos/pydata/xarray/issues/1068,582432731,MDEyOklzc3VlQ29tbWVudDU4MjQzMjczMQ==,1197350,2020-02-05T14:26:40Z,2020-02-05T14:26:40Z,MEMBER,"No, actually the problem was with my authorization. I had to accept a [EULA](https://urs.earthdata.nasa.gov/app_eula/nasa_gesdisc_data_archive) before my password was valid. Once I did that, everything worked. ","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-582236246,https://api.github.com/repos/pydata/xarray/issues/1068,582236246,MDEyOklzc3VlQ29tbWVudDU4MjIzNjI0Ng==,2448579,2020-02-05T04:35:00Z,2020-02-05T04:35:00Z,MEMBER,"https://en.wikipedia.org/wiki/HTTP_302 Looks like you need a better URL? and that pydap can't deal with redirects?","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-582226104,https://api.github.com/repos/pydata/xarray/issues/1068,582226104,MDEyOklzc3VlQ29tbWVudDU4MjIyNjEwNA==,1197350,2020-02-05T03:38:52Z,2020-02-05T03:38:52Z,MEMBER,"I am trying to load MERRA2 data via the NASA password-protected opendap server. Although it sounds like both pydap and xarray have been fixed to support this, I still am having basically the same problem @jenfly described over three years ago. At this point it feels like a pydap issue, but I ask on this thread anyway. Here's a fully reproducible example, password and all 😄 ```python from pydap.client import open_url from pydap.cas.urs import setup_session username = 'rabernat' password = '%8rTMU6VT37r&%3e' url = 'https://goldsmr5.gesdisc.eosdis.nasa.gov:443/opendap/MERRA2_MONTHLY/M2IMNPANA.5.12.4/2019/MERRA2_400.instM_3d_ana_Np.201901.nc4' session = setup_session(username, password, check_url=url) dataset = open_url(url, session=session) assert 'USVS' in dataset _ = dataset['USVS'][:] ``` raises ``` --------------------------------------------------------------------------- HTTPError Traceback (most recent call last) in ----> 1 _ = dataset['USVS'][:] /srv/conda/envs/notebook/lib/python3.7/site-packages/pydap/model.py in __getitem__(self, index) 318 def __getitem__(self, index): 319 out = copy.copy(self) --> 320 out.data = self._get_data_index(index) 321 return out 322 /srv/conda/envs/notebook/lib/python3.7/site-packages/pydap/model.py in _get_data_index(self, index) 347 return np.vectorize(decode_np_strings)(self._data[index]) 348 else: --> 349 return self._data[index] 350 351 def _get_data(self): /srv/conda/envs/notebook/lib/python3.7/site-packages/pydap/handlers/dap.py in __getitem__(self, index) 140 logger.info(""Fetching URL: %s"" % url) 141 r = GET(url, self.application, self.session, timeout=self.timeout) --> 142 raise_for_status(r) 143 dds, data = r.body.split(b'\nData:\n', 1) 144 dds = dds.decode(r.content_encoding or 'ascii') /srv/conda/envs/notebook/lib/python3.7/site-packages/pydap/net.py in raise_for_status(response) 37 detail=response.status+'\n'+response.text, 38 headers=response.headers, ---> 39 comment=response.body 40 ) 41 HTTPError: 302 Found 302 Found

Found

The document has moved here.

``` Is this a problem with pydap? Or the NASA server?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-327252508,https://api.github.com/repos/pydata/xarray/issues/1068,327252508,MDEyOklzc3VlQ29tbWVudDMyNzI1MjUwOA==,1217238,2017-09-05T17:46:54Z,2017-09-05T17:47:19Z,MEMBER,"@mrpgraae no, I don't think this has been implemented yet. Please take a look at #1508 for an example of the model to use: - Define an `open` classmethod method for loading from a URL. - `__init__` should accept a pydap dataset object (whatever is returned from `pydap.client.open_url`) You are also welcome to add any keyword parameters (e.g., `session`) that `open_url` accepts to the `open` method. So the user API becomes: ```python pydap_ds = pydap.client.open_url(url, session=session) store = xarray.backends.PydapDataStore(pydap_ds) ds = xarray.open_dataset(store) ``` or ```python store = xarray.backends.PydapDataStore.open(url, session=session) ds = xarray.open_dataset(store) ``` ","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-305946578,https://api.github.com/repos/pydata/xarray/issues/1068,305946578,MDEyOklzc3VlQ29tbWVudDMwNTk0NjU3OA==,1217238,2017-06-03T03:04:13Z,2017-06-03T03:04:13Z,MEMBER,"Nevermind, I figured it out (I was using an old version of pydap by mistake). See #1439 for the pydap fix.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-305830027,https://api.github.com/repos/pydata/xarray/issues/1068,305830027,MDEyOklzc3VlQ29tbWVudDMwNTgzMDAyNw==,1217238,2017-06-02T15:54:23Z,2017-06-02T15:54:23Z,MEMBER,I spent a few minutes on this but am still getting `AttributeError`. It would be great if someone could put some time into debugging this. Should be as simple as installing pydap (in both python 2 and 3 virtual/conda environments) and getting `py.test -k PydapTest` to pass.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-290879727,https://api.github.com/repos/pydata/xarray/issues/1068,290879727,MDEyOklzc3VlQ29tbWVudDI5MDg3OTcyNw==,1217238,2017-04-01T00:42:35Z,2017-04-01T00:42:35Z,MEMBER,"> I'm still having problems trying to get xarray.open_dataset to work with pydap. Using the latest commit on pydap/master (in which pydap/pydap#48 is merged) I'm now getting a new error: AttributeError: '' object has no attribute 'encode'. When I have some time, I'll look into it further and try to see what else is needed to restore compatibility. Indeed, it would be great if someone using pydap could take a look into this. You can find our logic for interoperating with pydap here: https://github.com/pydata/xarray/blob/master/xarray/backends/pydap_.py","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-288950068,https://api.github.com/repos/pydata/xarray/issues/1068,288950068,MDEyOklzc3VlQ29tbWVudDI4ODk1MDA2OA==,1217238,2017-03-24T07:14:10Z,2017-03-24T07:14:10Z,MEMBER,"Pydap has a new v3.2 release, but it still needs some fixes to work with xarray -- or xarray needs to be updated to work with the new version of pydap. I think https://github.com/pydap/pydap/pull/48 once merged would probably be enough to restore xarray compatibility.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-288948872,https://api.github.com/repos/pydata/xarray/issues/1068,288948872,MDEyOklzc3VlQ29tbWVudDI4ODk0ODg3Mg==,1217238,2017-03-24T07:06:00Z,2017-03-24T07:06:00Z,MEMBER,"Parsing username/password from the URL would be very easy to add. We need to figure out a solution for the proliferating arguments on `open_dataset` before we add many more, so I would prefer that for now. Another option is to add `session` as an argument on `xarray.backends.PydapDataStore`, and encourage passing `PydapDataStore` objects into `xarray.open_dataset` for extra customizability, e.g., ``` store = xarray.backends.PydapDataStore(url, session) ds = xarray.open_dataset(store) ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-287659092,https://api.github.com/repos/pydata/xarray/issues/1068,287659092,MDEyOklzc3VlQ29tbWVudDI4NzY1OTA5Mg==,1197350,2017-03-20T00:04:08Z,2017-03-20T00:04:08Z,MEMBER,"Hi @jenfly, it's great to see that you have tracked down this root issue! I agree we should be able to support direct access to these sort of opendap resources within xarray. It should not be too tricky to implement, and in fact, if you are interested, it could be a great opportunity for you to open a pull request and become directly involved in the project. We would be very happy to gain another contributor. You can see the line where `pydap.open_url` gets called here: https://github.com/pydata/xarray/blob/master/xarray/backends/pydap_.py#L64 We just need a mechanism to pass the username and password from `open_dataset` to the pydap backend. There are two possible options I see: 1. we could add new `username` and `password` keyword args to `open_dataset`. This is the most straightforward, but `open_dataset` already has a ton of arguments, so maybe it is not ideal. 1. we could parse out the username and password from a url like `https://username:password@...` within the pydap backend. It would be good to get some other opinions on which approach would be preferable.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-257485973,https://api.github.com/repos/pydata/xarray/issues/1068,257485973,MDEyOklzc3VlQ29tbWVudDI1NzQ4NTk3Mw==,1217238,2016-11-01T04:24:40Z,2016-11-01T04:24:40Z,MEMBER,"If the dataset has a ""time"" dimension, try accessing the first few values. Can you view them in pydap? Xarray's open_dataset does a little more work than pydap's open_url, insofar as it actually downloads some array data. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-257321771,https://api.github.com/repos/pydata/xarray/issues/1068,257321771,MDEyOklzc3VlQ29tbWVudDI1NzMyMTc3MQ==,1217238,2016-10-31T15:16:14Z,2016-10-31T15:16:14Z,MEMBER,"If you write `engine='pydap'` in open_dataset, the URL should be passed directly on to pydap, but you'll still need to follow all of their other instructions. If you're getting an error message from xarray, let us know but otherwise I'm at a loss -- you should check with the folks at NASA. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975