html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/7446#issuecomment-1385683582,https://api.github.com/repos/pydata/xarray/issues/7446,1385683582,IC_kwDOAMm_X85Sl9p-,1197350,2023-01-17T16:23:01Z,2023-01-17T16:23:01Z,MEMBER,"Hi @gauteh! This is very cool! Thanks for sharing. I'm really excited about way that Rust can be used to optimized different parts of our stack.
A couple of questions:
- Can your reader read over HTTP / S3 protocol? Or is it just local files?
- Do you know about [kerchunk](https://fsspec.github.io/kerchunk/)? The approach you described:
> The reader works by indexing the chunks of a dataset so that chunks can be accessed independently.
...is identical to the approach taken by Kerchunk (although the implementation is different). I'm curious what specification you use to store your indexes. Could we make your implementation interoperable with kerchunk, such that a kerchunk reference specification could be read by your reader? It would be great to reach for some degree of alignment here.
- Do you know about hdf5-coro - http://icesat2sliderule.org/h5coro/ - they have similar goals, but focused on cloud-based access
> I hope this can be of general interest, and if it would be of interest to move the hidefix xarray backend into xarray that would be very cool.
This is definitely of general interest! However, it is not necessary to add a new backend directly into xarray. We support entry points which allow packages to implement their own readers, as you have apparently already discovered: https://docs.xarray.dev/en/stable/internals/how-to-add-new-backend.html
Installing your package should be enough to enable the new engine.
We would, however, welcome a documentation PR that described how to use this package on the I/O page.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1536004355