html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue https://github.com/pydata/xarray/issues/1068#issuecomment-984936456,https://api.github.com/repos/pydata/xarray/issues/1068,984936456,IC_kwDOAMm_X846tPAI,717735,2021-12-02T19:30:14Z,2021-12-05T02:28:04Z,NONE,"I'm happy to help! @rabernat, Makhan @virdi from NASA Langley just reminded me the other day that there is a forum for NASA Earthdata users with a direct line to the program managers and scientists that may be the best place to ask data related questions. I think you only need to register with [EDL](https://urs.earthdata.nasa.gov/). https://forum.earthdata.nasa.gov/","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 1}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-984940677,https://api.github.com/repos/pydata/xarray/issues/1068,984940677,IC_kwDOAMm_X846tQCF,1197350,2021-12-02T19:36:12Z,2021-12-02T19:36:12Z,MEMBER,"One solution to this problem might be the creation of a [custom Xarray backend](http://xarray.pydata.org/en/stable/internals/how-to-add-new-backend.html) for NASA EarthData. This backend could manage authentication with EDL and have its own documentation. If this package were maintained by NASA, it would close the feedback loop more effectively.","{""total_count"": 5, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 4, ""eyes"": 1}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-984920867,https://api.github.com/repos/pydata/xarray/issues/1068,984920867,IC_kwDOAMm_X846tLMj,1197350,2021-12-02T19:08:54Z,2021-12-02T19:08:54Z,MEMBER,"Just wanted to say how much I appreciate @betolink acting as a communication channel between Xarray and NASA. Users often end up on our issue tracker because Xarray raises errors whenever it can't read data. But the source of these problems is not with Xarray, it's with the upstream data provider. This also happens all the time with xmitgcm, e.g. https://github.com/MITgcm/xmitgcm/issues/266 It would be great if NASA had a better way to respond to these issues which didn't require that you ""know a guy"".","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-971142092,https://api.github.com/repos/pydata/xarray/issues/1068,971142092,IC_kwDOAMm_X8454nPM,717735,2021-11-17T03:29:28Z,2021-11-17T03:29:28Z,NONE,"Quick update, MERRA2 worked as expected after accepting the EULA again. GPM_L3 redirects to an empty `.dods` file, I guess that's a bug. I'll ask the OpenDAP team tomorrow if they are aware of this behavior and what would be a workaround/solution.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-968999559,https://api.github.com/repos/pydata/xarray/issues/1068,968999559,IC_kwDOAMm_X845wcKH,717735,2021-11-15T15:04:37Z,2021-11-15T15:04:37Z,NONE,"I just asked on Slack about how to check for these changes (if at the end this issue is indeed related to an updated EULA) and unfortunately there is no way around it other than doing what Jan did(and still got the 302s). About feedback, yes there are channels but they are on a per-DAAC basis (cries). In this case that would be going to https://daac.gsfc.nasa.gov/ and clicking on the feedback button. I'll keep looking at this after the cloud hackathon today. ","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-968993065,https://api.github.com/repos/pydata/xarray/issues/1068,968993065,IC_kwDOAMm_X845wakp,1197350,2021-11-15T14:58:05Z,2021-11-15T14:58:05Z,MEMBER,At what point do we escalate this issue to NASA? Is there a channel via which they can receive and respond to user feedback?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-968986677,https://api.github.com/repos/pydata/xarray/issues/1068,968986677,IC_kwDOAMm_X845wZA1,717735,2021-11-15T14:51:28Z,2021-11-15T14:51:28Z,NONE,"Yeah, definitely not ideal. I'm going to test it again this evening with a new Earthdata user. I'll send you a binder link to a notebook to test it with both accounts.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-966777432,https://api.github.com/repos/pydata/xarray/issues/1068,966777432,IC_kwDOAMm_X845n9pY,717735,2021-11-12T02:49:59Z,2021-11-15T14:37:48Z,NONE,"Looks like the dataset got updated and when that happens NASA requires users to accept the end user license agreement (again). That's why the request ends up in a redirect. This EULA is also required the first time a user requests the data. Here are the instructions for accepting GESDISC EULA. https://disc.gsfc.nasa.gov/earthdata-login After the GESDIC data archive app shows up in our authorized apps list the code above works like a charm. I'll ask to see if there is a way to automate this @rabernat @zjans ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-968356690,https://api.github.com/repos/pydata/xarray/issues/1068,968356690,IC_kwDOAMm_X845t_NS,39352402,2021-11-14T20:18:29Z,2021-11-14T20:18:29Z,NONE,"@betolink Thanks for looking into this. GESDISC was already in my lists of accepted EULAS & authorized Apps. I also deleted them and re-authorized, but no change. I still get the ""302 The document has moved"" message when trying to access the HDF-datasets under https://gpm1.gesdisc.eosdis.nasa.gov/opendap/hyrax/GPM_L3/... with xr.backends.PydapDataStore and ds.open_dataset() In the meantime, I changed my scripts to download the entire HDF files from https://gpm1.gesdisc.eosdis.nasa.gov/data/GPM_L3/... and open them locally with xarray (and do spatial subsetting etc) - which works fine but is not quite ideal.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-966600850,https://api.github.com/repos/pydata/xarray/issues/1068,966600850,IC_kwDOAMm_X845nSiS,717735,2021-11-11T20:24:33Z,2021-11-11T20:24:33Z,NONE,This looks familiar. I'm going to take a look at this when I get home and will report back. @rabernat ,"{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-966324523,https://api.github.com/repos/pydata/xarray/issues/1068,966324523,IC_kwDOAMm_X845mPEr,1197350,2021-11-11T13:59:55Z,2021-11-11T13:59:55Z,MEMBER,"I'd like to tag @betolink in this issue. He knows quite a bit about both Xarray and Earthdata login. Maybe he can help us get to the bottom of these problems. Luis, any ideas?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-966198058,https://api.github.com/repos/pydata/xarray/issues/1068,966198058,IC_kwDOAMm_X845lwMq,29051639,2021-11-11T10:46:16Z,2021-11-11T10:46:16Z,CONTRIBUTOR,Unfortunately not @zjans ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-956786981,https://api.github.com/repos/pydata/xarray/issues/1068,956786981,IC_kwDOAMm_X845B2kl,39352402,2021-11-01T22:42:48Z,2021-11-01T22:42:48Z,NONE,"@AyrtonB I'm getting the same error now, did you manage to solve it?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-864477138,https://api.github.com/repos/pydata/xarray/issues/1068,864477138,MDEyOklzc3VlQ29tbWVudDg2NDQ3NzEzOA==,29051639,2021-06-19T23:51:09Z,2021-06-19T23:51:09Z,CONTRIBUTOR,"I'm also getting the same error when running `xr.open_dataset(store)` even though I have accepted the EULA. Has anyone had success solving this? I'm using pydap==3.2.2 and xarray==0.18.0, any help would be much appreciated! ```python import xarray as xr from pydap.client import open_url from pydap.cas.urs import setup_session username = ""my_username"" password= ""my_password"" url = 'https://goldsmr4.gesdisc.eosdis.nasa.gov/opendap/MERRA2/M2T1NXSLV.5.12.4/2016/06/MERRA2_400.tavg1_2d_slv_Nx.20160601.nc4' session = setup_session(username, password, check_url=url) pydap_ds = open_url(url, session=session) store = xr.backends.PydapDataStore(pydap_ds) ds = xr.open_dataset(store) ``` ```html HTTPError: 302 Found 302 Found

Found

The document has moved here.

```","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-790990219,https://api.github.com/repos/pydata/xarray/issues/1068,790990219,MDEyOklzc3VlQ29tbWVudDc5MDk5MDIxOQ==,71399442,2021-03-04T22:29:41Z,2021-03-04T22:29:41Z,NONE,@wallissoncarvalho Were you ever able to make that example work? I have been getting this error using the same example as well and haven't been able to find a solution ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-695812063,https://api.github.com/repos/pydata/xarray/issues/1068,695812063,MDEyOklzc3VlQ29tbWVudDY5NTgxMjA2Mw==,1519094,2020-09-20T17:16:08Z,2020-09-20T17:16:08Z,NONE,"Dear all, anyone knows if it is possible in xarray.open_dataset (pydap or netcdf engines) to pass `Authorization` or `Cookie` header along with opendap request? For example: `Authorization: Bearer u32t4o3tb3gg43` or `Cookie: foo=u32t4o3tb3gg43`","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-693648371,https://api.github.com/repos/pydata/xarray/issues/1068,693648371,MDEyOklzc3VlQ29tbWVudDY5MzY0ODM3MQ==,52915730,2020-09-16T20:30:12Z,2020-09-16T20:40:59Z,NONE,"> No, actually the problem was with my authorization. I had to accept a [EULA](https://urs.earthdata.nasa.gov/app_eula/nasa_gesdisc_data_archive) before my password was valid. Once I did that, everything worked. I'm trying this example: ```python url = 'https://gpm1.gesdisc.eosdis.nasa.gov:443/opendap/hyrax/GPM_L3/GPM_3IMERGHH.06/2019/087/3B-HHR.MS.MRG.3IMERG.20190328-S000000-E002959.0000.V06B.HDF5' try: session = setup_session(username, password, check_url=url) pydap_ds = open_url(url, session=session) store = xr.backends.PydapDataStore(pydap_ds) ds = xr.open_dataset(store) except Exception as err: print(err) ``` which returns: ``` 302 Found 302 Found

Found

The document has moved here.

/usr/local/lib/python3.8/site-packages/xarray/backends/common.py:87: FutureWarning: The ``variables`` property has been deprecated and will be removed in xarray v0.11. return len(self.variables) ``` The error message just comes when I try to use xr.open_dataset I've already accepted the EULA. Does anyone know what can be?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-624226394,https://api.github.com/repos/pydata/xarray/issues/1068,624226394,MDEyOklzc3VlQ29tbWVudDYyNDIyNjM5NA==,1197350,2020-05-05T18:23:57Z,2020-05-05T18:23:57Z,MEMBER,"> So, I tried Ryan's example, and got to the same error, where do you accept the EULA? It doesn't pop up on screen. https://urs.earthdata.nasa.gov/app_eula/nasa_gesdisc_data_archive","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-603979341,https://api.github.com/repos/pydata/xarray/issues/1068,603979341,MDEyOklzc3VlQ29tbWVudDYwMzk3OTM0MQ==,43081490,2020-03-25T17:31:39Z,2020-03-25T17:31:39Z,NONE,"So, I tried Ryan's example, and got to the same error, where do you accept the EULA? It doesn't pop up on screen. El dom., 22 mar. 2020 a las 6:29, ahahmann () escribió: > One can also add username and password to the .netrc file and all works > very smoothly, without a need for explicit username and password in the > script. > > However, there was one more issue. With Python 3.7.6, I was getting the > following error: > > File ""MERRA2.py"", line 16, in > session = setup_session(username, password, check_url=url) > File ""/groups/FutureWind/xesmf_env/lib/python3.7/site-packages/pydap/cas/urs.py"", line 19, in setup_session > verify=verify) > File ""/groups/FutureWind/xesmf_env/lib/python3.7/site-packages/pydap/cas/get_cookies.py"", line 75, in setup_session > password_field=password_field) > File ""/groups/FutureWind/xesmf_env/lib/python3.7/site-packages/pydap/cas/get_cookies.py"", line 123, in soup_login > soup = BeautifulSoup(resp.content, 'lxml') > File ""/groups/FutureWind/xesmf_env/lib/python3.7/site-packages/bs4/__init__.py"", line 228, in __init__ > % "","".join(features)) > bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: lxml. Do you need to install a parser library? > > That was solved by pip install lxml > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > , or > unsubscribe > > . > ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-602170564,https://api.github.com/repos/pydata/xarray/issues/1068,602170564,MDEyOklzc3VlQ29tbWVudDYwMjE3MDU2NA==,50208937,2020-03-22T09:29:18Z,2020-03-22T09:29:18Z,NONE,"One can also add username and password to the `.netrc` file and all works very smoothly, without a need for explicit username and password in the script. However, there was one more issue. With Python 3.7.6, I was getting the following error: ```Traceback (most recent call last): File ""MERRA2.py"", line 16, in session = setup_session(username, password, check_url=url) File ""/groups/FutureWind/xesmf_env/lib/python3.7/site-packages/pydap/cas/urs.py"", line 19, in setup_session verify=verify) File ""/groups/FutureWind/xesmf_env/lib/python3.7/site-packages/pydap/cas/get_cookies.py"", line 75, in setup_session password_field=password_field) File ""/groups/FutureWind/xesmf_env/lib/python3.7/site-packages/pydap/cas/get_cookies.py"", line 123, in soup_login soup = BeautifulSoup(resp.content, 'lxml') File ""/groups/FutureWind/xesmf_env/lib/python3.7/site-packages/bs4/__init__.py"", line 228, in __init__ % "","".join(features)) bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: lxml. Do you need to install a parser library? ``` That was solved by `pip install lxml`","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-582432731,https://api.github.com/repos/pydata/xarray/issues/1068,582432731,MDEyOklzc3VlQ29tbWVudDU4MjQzMjczMQ==,1197350,2020-02-05T14:26:40Z,2020-02-05T14:26:40Z,MEMBER,"No, actually the problem was with my authorization. I had to accept a [EULA](https://urs.earthdata.nasa.gov/app_eula/nasa_gesdisc_data_archive) before my password was valid. Once I did that, everything worked. ","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-582288134,https://api.github.com/repos/pydata/xarray/issues/1068,582288134,MDEyOklzc3VlQ29tbWVudDU4MjI4ODEzNA==,3404817,2020-02-05T08:06:28Z,2020-02-05T08:06:28Z,CONTRIBUTOR,"Yes, seems like a redirect issue. The URL is fine.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-582236246,https://api.github.com/repos/pydata/xarray/issues/1068,582236246,MDEyOklzc3VlQ29tbWVudDU4MjIzNjI0Ng==,2448579,2020-02-05T04:35:00Z,2020-02-05T04:35:00Z,MEMBER,"https://en.wikipedia.org/wiki/HTTP_302 Looks like you need a better URL? and that pydap can't deal with redirects?","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-582226104,https://api.github.com/repos/pydata/xarray/issues/1068,582226104,MDEyOklzc3VlQ29tbWVudDU4MjIyNjEwNA==,1197350,2020-02-05T03:38:52Z,2020-02-05T03:38:52Z,MEMBER,"I am trying to load MERRA2 data via the NASA password-protected opendap server. Although it sounds like both pydap and xarray have been fixed to support this, I still am having basically the same problem @jenfly described over three years ago. At this point it feels like a pydap issue, but I ask on this thread anyway. Here's a fully reproducible example, password and all 😄 ```python from pydap.client import open_url from pydap.cas.urs import setup_session username = 'rabernat' password = '%8rTMU6VT37r&%3e' url = 'https://goldsmr5.gesdisc.eosdis.nasa.gov:443/opendap/MERRA2_MONTHLY/M2IMNPANA.5.12.4/2019/MERRA2_400.instM_3d_ana_Np.201901.nc4' session = setup_session(username, password, check_url=url) dataset = open_url(url, session=session) assert 'USVS' in dataset _ = dataset['USVS'][:] ``` raises ``` --------------------------------------------------------------------------- HTTPError Traceback (most recent call last) in ----> 1 _ = dataset['USVS'][:] /srv/conda/envs/notebook/lib/python3.7/site-packages/pydap/model.py in __getitem__(self, index) 318 def __getitem__(self, index): 319 out = copy.copy(self) --> 320 out.data = self._get_data_index(index) 321 return out 322 /srv/conda/envs/notebook/lib/python3.7/site-packages/pydap/model.py in _get_data_index(self, index) 347 return np.vectorize(decode_np_strings)(self._data[index]) 348 else: --> 349 return self._data[index] 350 351 def _get_data(self): /srv/conda/envs/notebook/lib/python3.7/site-packages/pydap/handlers/dap.py in __getitem__(self, index) 140 logger.info(""Fetching URL: %s"" % url) 141 r = GET(url, self.application, self.session, timeout=self.timeout) --> 142 raise_for_status(r) 143 dds, data = r.body.split(b'\nData:\n', 1) 144 dds = dds.decode(r.content_encoding or 'ascii') /srv/conda/envs/notebook/lib/python3.7/site-packages/pydap/net.py in raise_for_status(response) 37 detail=response.status+'\n'+response.text, 38 headers=response.headers, ---> 39 comment=response.body 40 ) 41 HTTPError: 302 Found 302 Found

Found

The document has moved here.

``` Is this a problem with pydap? Or the NASA server?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-517231392,https://api.github.com/repos/pydata/xarray/issues/1068,517231392,MDEyOklzc3VlQ29tbWVudDUxNzIzMTM5Mg==,3639582,2019-08-01T10:47:12Z,2019-08-01T10:47:12Z,CONTRIBUTOR,"@juliancanellas Great! Good to see that someone else actually benefits from this feature, years after it was implemented 😄 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-516479160,https://api.github.com/repos/pydata/xarray/issues/1068,516479160,MDEyOklzc3VlQ29tbWVudDUxNjQ3OTE2MA==,43081490,2019-07-30T15:49:55Z,2019-07-30T15:49:55Z,NONE,"Dear all, Thank you very much for all the time you've put into fixing this issue. I'm a fresh PhD student, started working on solar radiation forecast four months ago, and right now I'm trying to use MERRA 2 aerosol data to initialize WRF Solar. The bug fix on this thread has helped me a lot, since I was trying to avoid the straight forward method of downloading the files by date and then merging them in a single python object. This way I can directly create my python object without downloading one by one and then merging! It's awesome! Thank you all very much! ","{""total_count"": 3, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 3, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-327413829,https://api.github.com/repos/pydata/xarray/issues/1068,327413829,MDEyOklzc3VlQ29tbWVudDMyNzQxMzgyOQ==,3639582,2017-09-06T08:30:22Z,2017-09-06T08:30:22Z,CONTRIBUTOR,"Thank you @shoyer, I'll start work on the implementation.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-327252508,https://api.github.com/repos/pydata/xarray/issues/1068,327252508,MDEyOklzc3VlQ29tbWVudDMyNzI1MjUwOA==,1217238,2017-09-05T17:46:54Z,2017-09-05T17:47:19Z,MEMBER,"@mrpgraae no, I don't think this has been implemented yet. Please take a look at #1508 for an example of the model to use: - Define an `open` classmethod method for loading from a URL. - `__init__` should accept a pydap dataset object (whatever is returned from `pydap.client.open_url`) You are also welcome to add any keyword parameters (e.g., `session`) that `open_url` accepts to the `open` method. So the user API becomes: ```python pydap_ds = pydap.client.open_url(url, session=session) store = xarray.backends.PydapDataStore(pydap_ds) ds = xarray.open_dataset(store) ``` or ```python store = xarray.backends.PydapDataStore.open(url, session=session) ds = xarray.open_dataset(store) ``` ","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-327179289,https://api.github.com/repos/pydata/xarray/issues/1068,327179289,MDEyOklzc3VlQ29tbWVudDMyNzE3OTI4OQ==,3639582,2017-09-05T13:43:55Z,2017-09-05T13:43:55Z,CONTRIBUTOR,"@shoyer @jenfly Has this been implemented? I can't see any open PRs relating to this, so I guess no one is working on it? I would be happy to try and implement it, if that's fine with you? It seems like you settled on the solution of passing a session object to a PydapDataStore and then passing that to open_dataset(), correct? Thanks in advance!","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-305946578,https://api.github.com/repos/pydata/xarray/issues/1068,305946578,MDEyOklzc3VlQ29tbWVudDMwNTk0NjU3OA==,1217238,2017-06-03T03:04:13Z,2017-06-03T03:04:13Z,MEMBER,"Nevermind, I figured it out (I was using an old version of pydap by mistake). See #1439 for the pydap fix.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-305830027,https://api.github.com/repos/pydata/xarray/issues/1068,305830027,MDEyOklzc3VlQ29tbWVudDMwNTgzMDAyNw==,1217238,2017-06-02T15:54:23Z,2017-06-02T15:54:23Z,MEMBER,I spent a few minutes on this but am still getting `AttributeError`. It would be great if someone could put some time into debugging this. Should be as simple as installing pydap (in both python 2 and 3 virtual/conda environments) and getting `py.test -k PydapTest` to pass.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-305775273,https://api.github.com/repos/pydata/xarray/issues/1068,305775273,MDEyOklzc3VlQ29tbWVudDMwNTc3NTI3Mw==,3217406,2017-06-02T12:44:23Z,2017-06-02T13:00:46Z,CONTRIBUTOR,@jenfly and @shoyer ``pydap`` version 3.2.2 (newly released last week) should have fixed this issue. Could you verify?,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-291410815,https://api.github.com/repos/pydata/xarray/issues/1068,291410815,MDEyOklzc3VlQ29tbWVudDI5MTQxMDgxNQ==,13774246,2017-04-04T06:54:34Z,2017-04-04T06:54:34Z,NONE,"Awesome, thanks so much @laliberte!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-291262663,https://api.github.com/repos/pydata/xarray/issues/1068,291262663,MDEyOklzc3VlQ29tbWVudDI5MTI2MjY2Mw==,3217406,2017-04-03T20:24:07Z,2017-04-03T20:24:07Z,CONTRIBUTOR,"@shoyer @jenfly: Good news, I think I was able to track down the bug in ``pydap`` that was preventing compatibility. I'm putting a PR together and we could expect it to be merged pretty soon into the master. I wanted to give you a heads up so that you don't waste more time on this.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-290879727,https://api.github.com/repos/pydata/xarray/issues/1068,290879727,MDEyOklzc3VlQ29tbWVudDI5MDg3OTcyNw==,1217238,2017-04-01T00:42:35Z,2017-04-01T00:42:35Z,MEMBER,"> I'm still having problems trying to get xarray.open_dataset to work with pydap. Using the latest commit on pydap/master (in which pydap/pydap#48 is merged) I'm now getting a new error: AttributeError: '' object has no attribute 'encode'. When I have some time, I'll look into it further and try to see what else is needed to restore compatibility. Indeed, it would be great if someone using pydap could take a look into this. You can find our logic for interoperating with pydap here: https://github.com/pydata/xarray/blob/master/xarray/backends/pydap_.py","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-290870420,https://api.github.com/repos/pydata/xarray/issues/1068,290870420,MDEyOklzc3VlQ29tbWVudDI5MDg3MDQyMA==,13774246,2017-03-31T23:53:20Z,2017-03-31T23:53:20Z,NONE,"I also like the idea of passing `PydapDataStore` objects that include the `session` object. Delving deeper into the `pydap` authentication, I found that there are already several different `setup_session` functions available to create the `session` object, corresponding to different authentication procedures (`pydap.cas.get_cookies.setup_session`, `pydap.cas.urs.setup_session`, `pydap.cas.esgf.setup_session`) as well as additional arguments to `setup_session` beyond `username` and `password`. Best to deal with all this separately with `pydap` rather than trying to embed it within `xarray`. I'm still having problems trying to get `xarray.open_dataset` to work with `pydap`. Using the latest commit on `pydap/master` (in which https://github.com/pydap/pydap/pull/48 is merged) I'm now getting a new error: `AttributeError: '' object has no attribute 'encode'`. When I have some time, I'll look into it further and try to see what else is needed to restore compatibility.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-290189772,https://api.github.com/repos/pydata/xarray/issues/1068,290189772,MDEyOklzc3VlQ29tbWVudDI5MDE4OTc3Mg==,3217406,2017-03-29T18:54:09Z,2017-03-29T18:54:09Z,CONTRIBUTOR,"I like the idea of passing ``PydapDataStore`` objects that include the ``session`` object. It seems more likely to be forward compatible, especially if Central Authentication Services multiply (as one would expect) with different authentication mechanisms. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-288950068,https://api.github.com/repos/pydata/xarray/issues/1068,288950068,MDEyOklzc3VlQ29tbWVudDI4ODk1MDA2OA==,1217238,2017-03-24T07:14:10Z,2017-03-24T07:14:10Z,MEMBER,"Pydap has a new v3.2 release, but it still needs some fixes to work with xarray -- or xarray needs to be updated to work with the new version of pydap. I think https://github.com/pydap/pydap/pull/48 once merged would probably be enough to restore xarray compatibility.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-288948872,https://api.github.com/repos/pydata/xarray/issues/1068,288948872,MDEyOklzc3VlQ29tbWVudDI4ODk0ODg3Mg==,1217238,2017-03-24T07:06:00Z,2017-03-24T07:06:00Z,MEMBER,"Parsing username/password from the URL would be very easy to add. We need to figure out a solution for the proliferating arguments on `open_dataset` before we add many more, so I would prefer that for now. Another option is to add `session` as an argument on `xarray.backends.PydapDataStore`, and encourage passing `PydapDataStore` objects into `xarray.open_dataset` for extra customizability, e.g., ``` store = xarray.backends.PydapDataStore(url, session) ds = xarray.open_dataset(store) ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-287670125,https://api.github.com/repos/pydata/xarray/issues/1068,287670125,MDEyOklzc3VlQ29tbWVudDI4NzY3MDEyNQ==,13774246,2017-03-20T02:54:33Z,2017-03-20T02:54:33Z,NONE,"Thanks, @rabernat! I'd be happy to try implementing this in the project. I'm a newbie when it comes to contributing to big projects like this (so far I've just used Github for my own little projects) so I might have some naive questions as I figure out how things work. The two options you mentioned for passing username and password info to `open_dataset` both sound good to me. I don't have any strong preference between them. How do I get other opinions on which approach to use? Should I start a new issue thread? Also, I realized that there is another hiccup along the way. When I try to specify `engine='pydap'` in `open_dataset`, I get the same error message as mentioned in #1174, that the object has no attribute `iteritems`. When I wrote the first post in this thread, back in October, I was able to use `engine='pydap'` without any problems. This seems to be related to recent upstream changes in Pydap: https://github.com/pydap/pydap/issues/43 and I presume might require more substantial changes either in Xarray or Pydap so that they can work together again. Any thoughts on how to handle this?","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-287659092,https://api.github.com/repos/pydata/xarray/issues/1068,287659092,MDEyOklzc3VlQ29tbWVudDI4NzY1OTA5Mg==,1197350,2017-03-20T00:04:08Z,2017-03-20T00:04:08Z,MEMBER,"Hi @jenfly, it's great to see that you have tracked down this root issue! I agree we should be able to support direct access to these sort of opendap resources within xarray. It should not be too tricky to implement, and in fact, if you are interested, it could be a great opportunity for you to open a pull request and become directly involved in the project. We would be very happy to gain another contributor. You can see the line where `pydap.open_url` gets called here: https://github.com/pydata/xarray/blob/master/xarray/backends/pydap_.py#L64 We just need a mechanism to pass the username and password from `open_dataset` to the pydap backend. There are two possible options I see: 1. we could add new `username` and `password` keyword args to `open_dataset`. This is the most straightforward, but `open_dataset` already has a ton of arguments, so maybe it is not ideal. 1. we could parse out the username and password from a url like `https://username:password@...` within the pydap backend. It would be good to get some other opinions on which approach would be preferable.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-287653485,https://api.github.com/repos/pydata/xarray/issues/1068,287653485,MDEyOklzc3VlQ29tbWVudDI4NzY1MzQ4NQ==,13774246,2017-03-19T22:32:50Z,2017-03-19T22:36:08Z,NONE,"I've finally found something useful online and am able to use Pydap to open these files -- hoping someone can help me find a way to integrate this into an xarray.open_dataset() function call and then I will be a very happy camper! Turns out much of the info posted by NASA online is out of date and based on a different implementation of Pydap than what is actually being used currently (argh). Here is something that actually works, from http://www.pydap.org/en/latest/client.html#urs-nasa-earthdata: ``` from pydap.client import open_url from pydap.cas.urs import setup_session url = 'https://goldsmr4.gesdisc.eosdis.nasa.gov/opendap/MERRA2/M2T1NXSLV.5.12.4/2016/06/MERRA2_400.tavg1_2d_slv_Nx.20160601.nc4' session = setup_session(username, password) dataset = open_url(url, session=session) ``` where I've assigned the username and password variables with the appropriate values in another function. I've tested this and it is working, but I would prefer to do things within Xarray since all my code is already using it. Just for fun, I tried `ds = xarray.open_dataset(url, engine='pydap', session=session)`, to see if the extra keyword would be magically sent to the pydap engine, but got an error message. Is there a way to incorporate this functionality into xarray.open_dataset? Thank you so much for any assistance!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-263029510,https://api.github.com/repos/pydata/xarray/issues/1068,263029510,MDEyOklzc3VlQ29tbWVudDI2MzAyOTUxMA==,13774246,2016-11-25T22:41:39Z,2016-11-25T22:41:39Z,NONE,"@j08lue no, not yet. I've been in touch with the folks at NASA who run the server, but their suggestions didn't work for me and I haven't had time to keep troubleshooting. I will need to sort out this issue in the next couple of months to get some data that I need, so if/when I ever resolve it, I'll post the solution here.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-263007655,https://api.github.com/repos/pydata/xarray/issues/1068,263007655,MDEyOklzc3VlQ29tbWVudDI2MzAwNzY1NQ==,3404817,2016-11-25T18:21:05Z,2016-11-25T18:21:05Z,CONTRIBUTOR,"@jenfly did you find a solution how to make opendap authentication work with `xarray`? Might be worthwhile posting it here, even though the issue has to do with the backends.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-258002779,https://api.github.com/repos/pydata/xarray/issues/1068,258002779,MDEyOklzc3VlQ29tbWVudDI1ODAwMjc3OQ==,13774246,2016-11-02T21:20:52Z,2016-11-02T21:20:52Z,NONE,"Ah, I see. Thanks for the suggestion. Using Pydap I'm able to see all the variables and their metadata, so I thought it was working, but when I try to actually access the data values, I get the same error message as from Xarray. The issue must be something unrelated to Xarray -- I'll keep investigating. Thanks for your help! ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-257485973,https://api.github.com/repos/pydata/xarray/issues/1068,257485973,MDEyOklzc3VlQ29tbWVudDI1NzQ4NTk3Mw==,1217238,2016-11-01T04:24:40Z,2016-11-01T04:24:40Z,MEMBER,"If the dataset has a ""time"" dimension, try accessing the first few values. Can you view them in pydap? Xarray's open_dataset does a little more work than pydap's open_url, insofar as it actually downloads some array data. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-257479761,https://api.github.com/repos/pydata/xarray/issues/1068,257479761,MDEyOklzc3VlQ29tbWVudDI1NzQ3OTc2MQ==,13774246,2016-11-01T03:15:46Z,2016-11-01T03:15:46Z,NONE,"Thanks very much for your reply! I still get an error from xarray when I use the `engine='pydap'` option. Here's a minimum (almost) working example (almost because you need an account with the server so you can substitute your username/password into the url string): ``` import xarray from pydap.client import open_url url = 'http://:@goldsmr5.sci.gsfc.nasa.gov/opendap/MERRA2/M2I3NPASM.5.12.4/1986/01/MERRA2_100.inst3_3d_asm_Np.19860101.nc4' ds1 = open_url(url) # Works but data isn't in xarray format ds2 = xarray.open_dataset(url, engine='pydap') # Error message, see attached ``` I've attached the error message here -- [error_msg.txt](https://github.com/pydata/xarray/files/563281/error_msg.txt) I don't know enough about the inner workings of xarray to trace through it. Please let me know if any of this means anything to you and has a reasonably easy fix or workaround. Thank you! ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975 https://github.com/pydata/xarray/issues/1068#issuecomment-257321771,https://api.github.com/repos/pydata/xarray/issues/1068,257321771,MDEyOklzc3VlQ29tbWVudDI1NzMyMTc3MQ==,1217238,2016-10-31T15:16:14Z,2016-10-31T15:16:14Z,MEMBER,"If you write `engine='pydap'` in open_dataset, the URL should be passed directly on to pydap, but you'll still need to follow all of their other instructions. If you're getting an error message from xarray, let us know but otherwise I'm at a loss -- you should check with the folks at NASA. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186169975