html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue https://github.com/pydata/xarray/issues/1854#issuecomment-364494085,https://api.github.com/repos/pydata/xarray/issues/1854,364494085,MDEyOklzc3VlQ29tbWVudDM2NDQ5NDA4NQ==,1197350,2018-02-09T17:03:06Z,2018-02-09T17:03:06Z,MEMBER,"@jhamman, chunking in lat and lon should not be necessary here. My understanding is that dask/dask#2364 made sure that the indexing operation happens before the concat. One possibility is that the files have HDF-level chunking / compression, as discussed in #1440. That could be screwing this up.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,291332965 https://github.com/pydata/xarray/issues/1854#issuecomment-364490209,https://api.github.com/repos/pydata/xarray/issues/1854,364490209,MDEyOklzc3VlQ29tbWVudDM2NDQ5MDIwOQ==,1197350,2018-02-09T16:50:13Z,2018-02-09T16:50:13Z,MEMBER,"Also, maybe you can post this dataset somewhere online for us to play around with?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,291332965 https://github.com/pydata/xarray/issues/1854#issuecomment-364489976,https://api.github.com/repos/pydata/xarray/issues/1854,364489976,MDEyOklzc3VlQ29tbWVudDM2NDQ4OTk3Ng==,1197350,2018-02-09T16:49:30Z,2018-02-09T16:49:30Z,MEMBER,Did you try my workaround?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,291332965 https://github.com/pydata/xarray/issues/1854#issuecomment-364465016,https://api.github.com/repos/pydata/xarray/issues/1854,364465016,MDEyOklzc3VlQ29tbWVudDM2NDQ2NTAxNg==,1197350,2018-02-09T15:26:40Z,2018-02-09T15:26:40Z,MEMBER,"The way this *should* work is that the selection of a single point should happen *before* the data is concatenated. It is up to dask to properly ""fuse"" these two operations. It seems like that is failing for some reason. As a temporary workaround, you could preprocess the data to only select the specific point *before* concatenating. ```python def select_point(ds): return ds.sel(latitude=10, longitude=10) ds = xr.open_mfdataset('*.nc', preprocesses=select_point) ``` But you shouldn't have to do this to get good performance here.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,291332965 https://github.com/pydata/xarray/issues/1854#issuecomment-364462150,https://api.github.com/repos/pydata/xarray/issues/1854,364462150,MDEyOklzc3VlQ29tbWVudDM2NDQ2MjE1MA==,1197350,2018-02-09T15:16:54Z,2018-02-09T15:16:54Z,MEMBER,"This sounds similar to #1396, which I thought was resolved (but is still marked as open).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,291332965 https://github.com/pydata/xarray/issues/1854#issuecomment-364461729,https://api.github.com/repos/pydata/xarray/issues/1854,364461729,MDEyOklzc3VlQ29tbWVudDM2NDQ2MTcyOQ==,1197350,2018-02-09T15:15:28Z,2018-02-09T15:15:28Z,MEMBER,Can you just try your full example without the `chunks` argument and see if it works any better?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,291332965 https://github.com/pydata/xarray/issues/1854#issuecomment-364456174,https://api.github.com/repos/pydata/xarray/issues/1854,364456174,MDEyOklzc3VlQ29tbWVudDM2NDQ1NjE3NA==,1197350,2018-02-09T14:56:36Z,2018-02-09T14:56:36Z,MEMBER,"No, I meant this: ```python ds = xr.open_mfdataset('path/to/ncs/*.nc', chunks={'time': 127}) ds_point = ds.sel(latitude=10, longitude=10) repr(ds_point) ``` Also, your comment says that ""127 is normally the size of the time dimension in each file"", but the info you posted indicates that it's 248. Can you also try `open_mfsdataset` without the `chunks` argument?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,291332965 https://github.com/pydata/xarray/issues/1854#issuecomment-364447957,https://api.github.com/repos/pydata/xarray/issues/1854,364447957,MDEyOklzc3VlQ29tbWVudDM2NDQ0Nzk1Nw==,1197350,2018-02-09T14:26:46Z,2018-02-09T14:26:46Z,MEMBER,"I am puzzled by this. Selecting a single point should not require loading into memory the whole dataset. Can you post the output of `repr(ds.sel(latitude=10, longitude=10))`?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,291332965 https://github.com/pydata/xarray/issues/1854#issuecomment-360779298,https://api.github.com/repos/pydata/xarray/issues/1854,360779298,MDEyOklzc3VlQ29tbWVudDM2MDc3OTI5OA==,1197350,2018-01-26T13:04:31Z,2018-01-26T13:04:31Z,MEMBER,"Can you provide a bit more info about the structure of the individual files? Open a single file and call ds.info(), then paste the contents here. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,291332965