home / github / issue_comments

Menu
  • Search all tables
  • GraphQL API

issue_comments: 223905394

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/463#issuecomment-223905394 https://api.github.com/repos/pydata/xarray/issues/463 223905394 MDEyOklzc3VlQ29tbWVudDIyMzkwNTM5NA== 743508 2016-06-06T09:06:33Z 2016-06-06T09:06:33Z CONTRIBUTOR

@shoyer thanks - here's how i'm using mfdataset - not using any options. I'm going to try using the h5netcdf backend to see if I get the same results. I'm still not 100% confident that I'm tracking open files correctly with lsof so I'm going to try to make a minimal example to investigate.

``` python

def weather_dataset(root_path: Path, *, start_date: datetime = None, end_date: datetime = None): flat_files_paths = get_dset_file_paths(root_path, start_date=start_date, end_date=end_date) # Convert Paths to list of strings for xarray dataset = xr.open_mfdataset([str(f) for f in flat_files_paths]) return dataset

def cfsr_weather_loader(db, site_lookup_fn=None, dset_start=None, dset_end=None, site_conf=None): # Pull values out of the dt_conf = site_conf if site_conf else WEATHER_CFSR dset_start = dset_start if dset_start else dt_conf['start_dt'] dset_end = dset_end if dset_end else dt_conf['end_dt']

if site_lookup_fn is None:
    site_lookup_fn = site_lookup_postcode_district

def weather_loader(site_id, start_date, end_date, resample=None):
    # using the tuple because always getting mixed up with lon/lat
    geo_lookup = site_lookup_fn(site_id, db)

    # With statement should ensure dset is closed after loading.
    with weather_dataset(WEATHER_CFSR['path'],
                         start_date=dset_start,
                         end_date=dset_end) as weather:
        data = weighted_regional_timeseries(weather, start_date, end_date,
                                            lon=geo_lookup.lon,
                                            lat=geo_lookup.lat,
                                            weights=geo_lookup.weights)

    # RENAME from CFSR standard
    data = data.rename(columns=WEATHER_RENAME)

    if resample is not None:
        data = data.resample(resample).mean()
    data.irradiance /= 1000.0  # convert irradiance to kW
    return data

return weather_loader

```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  94328498
Powered by Datasette · Queries took 0.73ms · About: xarray-datasette