issues: 449706080
This data as json
| id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | active_lock_reason | draft | pull_request | body | reactions | performed_via_github_app | state_reason | repo | type |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 449706080 | MDU6SXNzdWU0NDk3MDYwODA= | 2995 | Remote writing NETCDF4 files to Amazon S3 | 51129665 | open | 0 | 10 | 2019-05-29T09:46:47Z | 2023-04-20T16:38:46Z | NONE | Hi all, I am trying to use xarray's .to_netcdf() function to write an array remotely to a file in an Amazon S3 bucket in NETCDF4 format, but I have absolutely no idea how to do it. So far I have been able to write locally, but it all completely falls apart when I try and provide something that isn't a local path to the .to_netcdf() function. Example code is provided below. import boto3 import json import numpy as np import xarray as xr with open('<MY_PATH>/boto_test_credentials', 'r') as f: secrets = json.load(f) sn = secrets['service_name'] aaki = secrets['aws_access_key_id'] asak = secrets['aws_secret_access_key'] eu = secrets['endpoint_url'] session = boto3.session.Session() s3 = session.client( service_name = sn, aws_access_key_id = aaki, aws_secret_access_key = asak, endpoint_url = eu, ) x = xr.DataArray(np.random.randn(450, 450)) Then finally something like this? x.to_netcdf(<URL_TO_MY_AMAZON_S3_BUCKET>) I've thought about trying to trick the .to_netcdf() function by using the urllib module to convert a URL into a file-like object, but I don't understand thing like the authentication process etc well enough, and I don't even know if I'm heading down a blind alley here. Any advice would be most appreciated! |
{
"url": "https://api.github.com/repos/pydata/xarray/issues/2995/reactions",
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
13221727 | issue |