html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/2501#issuecomment-432546977,https://api.github.com/repos/pydata/xarray/issues/2501,432546977,MDEyOklzc3VlQ29tbWVudDQzMjU0Njk3Nw==,1492047,2018-10-24T07:38:31Z,2018-10-24T07:38:31Z,CONTRIBUTOR,"Thank you for looking into this.
I just want to point out that I'm not that much concerned with the ""slow performance"" but much more with the memory consumption and the limitation it implies.
```python
from glob import glob
import xarray as xr
all_files = glob('...*TP110*.nc')
display(xr.open_dataset(all_files[0]))
display(xr.open_dataset(all_files[1]))
```
```
<xarray.Dataset>
Dimensions: (meas_ind: 40, time: 2871, wvf_ind: 128)
Coordinates:
* time (time) datetime64[ns] 2017-06-19T14:24:20.792036992 ... 2017-06-19T15:14:38.491743104
* meas_ind (meas_ind) int8 0 1 2 3 4 ... 36 37 38 39
* wvf_ind (wvf_ind) int8 0 1 2 3 ... 125 126 127
lat (time) float64 ...
lon (time) float64 ...
lon_40hz (time, meas_ind) float64 ...
lat_40hz (time, meas_ind) float64 ...
Data variables:
time_40hz (time, meas_ind) datetime64[ns] ...
surface_type (time) float32 ...
rad_surf_type (time) float32 ...
qual_alt_1hz_range (time) float32 ...
qual_alt_1hz_swh (time) float32 ...
qual_alt_1hz_sig0 (time) float32 ...
qual_alt_1hz_off_nadir_angle_wf (time) float32 ...
qual_inst_corr_1hz_range (time) float32 ...
qual_inst_corr_1hz_swh (time) float32 ...
qual_inst_corr_1hz_sig0 (time) float32 ...
qual_rad_1hz_tb_k (time) float32 ...
qual_rad_1hz_tb_ka (time) float32 ...
alt_state_flag_acq_mode_40hz (time, meas_ind) float32 ...
alt_state_flag_tracking_mode_40hz (time, meas_ind) float32 ...
orb_state_flag_diode (time) float32 ...
orb_state_flag_rest (time) float32 ...
ecmwf_meteo_map_avail (time) float32 ...
trailing_edge_variation_flag (time) float32 ...
trailing_edge_variation_flag_40hz (time, meas_ind) float32 ...
ice_flag (time) float32 ...
interp_flag_mean_sea_surface (time) float32 ...
interp_flag_mdt (time) float32 ...
interp_flag_ocean_tide_sol1 (time) float32 ...
interp_flag_ocean_tide_sol2 (time) float32 ...
interp_flag_meteo (time) float32 ...
alt (time) float64 ...
alt_40hz (time, meas_ind) float64 ...
orb_alt_rate (time) float32 ...
range (time) float64 ...
range_40hz (time, meas_ind) float64 ...
range_used_40hz (time, meas_ind) float32 ...
range_rms (time) float32 ...
range_numval (time) float32 ...
number_of_iterations (time, meas_ind) float32 ...
net_instr_corr_range (time) float64 ...
model_dry_tropo_corr (time) float32 ...
model_wet_tropo_corr (time) float32 ...
rad_wet_tropo_corr (time) float32 ...
iono_corr_gim (time) float32 ...
sea_state_bias (time) float32 ...
swh (time) float32 ...
swh_40hz (time, meas_ind) float32 ...
swh_used_40hz (time, meas_ind) float32 ...
swh_rms (time) float32 ...
swh_numval (time) float32 ...
net_instr_corr_swh (time) float32 ...
sig0 (time) float32 ...
sig0_40hz (time, meas_ind) float32 ...
sig0_used_40hz (time, meas_ind) float32 ...
sig0_rms (time) float32 ...
sig0_numval (time) float32 ...
agc (time) float32 ...
agc_rms (time) float32 ...
agc_numval (time) float32 ...
net_instr_corr_sig0 (time) float32 ...
atmos_corr_sig0 (time) float32 ...
off_nadir_angle_wf (time) float32 ...
off_nadir_angle_wf_40hz (time, meas_ind) float32 ...
tb_k (time) float32 ...
tb_ka (time) float32 ...
mean_sea_surface (time) float64 ...
mean_topography (time) float64 ...
geoid (time) float64 ...
bathymetry (time) float64 ...
inv_bar_corr (time) float32 ...
hf_fluctuations_corr (time) float32 ...
ocean_tide_sol1 (time) float64 ...
ocean_tide_sol2 (time) float64 ...
ocean_tide_equil (time) float32 ...
ocean_tide_non_equil (time) float32 ...
load_tide_sol1 (time) float32 ...
load_tide_sol2 (time) float32 ...
solid_earth_tide (time) float32 ...
pole_tide (time) float32 ...
wind_speed_model_u (time) float32 ...
wind_speed_model_v (time) float32 ...
wind_speed_alt (time) float32 ...
rad_water_vapor (time) float32 ...
rad_liquid_water (time) float32 ...
ice1_range_40hz (time, meas_ind) float64 ...
ice1_sig0_40hz (time, meas_ind) float32 ...
ice1_qual_flag_40hz (time, meas_ind) float32 ...
seaice_range_40hz (time, meas_ind) float64 ...
seaice_sig0_40hz (time, meas_ind) float32 ...
seaice_qual_flag_40hz (time, meas_ind) float32 ...
ice2_range_40hz (time, meas_ind) float64 ...
ice2_le_sig0_40hz (time, meas_ind) float32 ...
ice2_sig0_40hz (time, meas_ind) float32 ...
ice2_sigmal_40hz (time, meas_ind) float32 ...
ice2_slope1_40hz (time, meas_ind) float64 ...
ice2_slope2_40hz (time, meas_ind) float64 ...
ice2_mqe_40hz (time, meas_ind) float32 ...
ice2_qual_flag_40hz (time, meas_ind) float32 ...
mqe_40hz (time, meas_ind) float32 ...
peakiness_40hz (time, meas_ind) float32 ...
ssha (time) float32 ...
tracker_40hz (time, meas_ind) float64 ...
tracker_used_40hz (time, meas_ind) float32 ...
tracker_diode_40hz (time, meas_ind) float64 ...
pri_counter_40hz (time, meas_ind) float64 ...
qual_alt_1hz_off_nadir_angle_pf (time) float32 ...
off_nadir_angle_pf (time) float32 ...
off_nadir_angle_rain_40hz (time, meas_ind) float32 ...
uso_corr (time) float64 ...
internal_path_delay_corr (time) float64 ...
modeled_instr_corr_range (time) float32 ...
doppler_corr (time) float32 ...
cog_corr (time) float32 ...
modeled_instr_corr_swh (time) float32 ...
internal_corr_sig0 (time) float32 ...
modeled_instr_corr_sig0 (time) float32 ...
agc_40hz (time, meas_ind) float32 ...
agc_corr_40hz (time, meas_ind) float32 ...
scaling_factor_40hz (time, meas_ind) float64 ...
epoch_40hz (time, meas_ind) float64 ...
width_leading_edge_40hz (time, meas_ind) float64 ...
amplitude_40hz (time, meas_ind) float64 ...
thermal_noise_40hz (time, meas_ind) float64 ...
seaice_epoch_40hz (time, meas_ind) float64 ...
seaice_amplitude_40hz (time, meas_ind) float64 ...
ice2_epoch_40hz (time, meas_ind) float64 ...
ice2_amplitude_40hz (time, meas_ind) float64 ...
ice2_mean_amplitude_40hz (time, meas_ind) float64 ...
ice2_thermal_noise_40hz (time, meas_ind) float64 ...
ice2_slope_40hz (time, meas_ind) float64 ...
signal_to_noise_ratio (time) float32 ...
waveforms_40hz (time, meas_ind, wvf_ind) float32 ...
Attributes:
Conventions: CF-1.1
title: GDR - Expertise dataset
institution: CNES
source: radar altimeter
history: 2017-07-21 08:25:07 : Creation
contact: CNES aviso@oceanobs.com, EUMETSAT ops@...
references: L1 library=V4.5p1, L2 library=V5.5p2, ...
processing_center: SALP
reference_document: SARAL/ALTIKA Products Handbook, SALP-M...
mission_name: SARAL
altimeter_sensor_name: ALTIKA
radiometer_sensor_name: ALTIKA_RAD
doris_sensor_name: DGXX
cycle_number: 110
absolute_rev_number: 22545
pass_number: 1
absolute_pass_number: 109219
equator_time: 2017-06-19 14:49:32.128000
equator_longitude: 227.77
first_meas_time: 2017-06-19 14:24:20.792037
last_meas_time: 2017-06-19 15:14:38.491743
xref_altimeter_level1: ALK_ALT_1PaS20170619_154722_20170619_1...
xref_radiometer_level1: ALK_RAD_1PaS20170619_154643_20170619_1...
xref_altimeter_characterisation: ALK_CHA_AXVCNE20131115_120000_20100101...
xref_radiometer_characterisation: ALK_CHR_AXVCNE20110207_180000_20110101...
xref_altimeter_ltm: ALK_CAL_AXXCNE20170720_110014_20130102...
xref_doris_uso: SRL_OS1_AXXCNE20170720_083800_20130226...
xref_orbit_data: SRL_VOR_AXVCNE20170720_111700_20170618...
xref_pf_data: SRL_VPF_AXVCNE20170720_111800_20170618...
xref_pole_location: SMM_POL_AXXCNE20170721_071500_19870101...
xref_gim_data: SRL_ION_AXPCNE20170620_074756_20170619...
xref_mog2d_data: SMM_MOG_AXVCNE20170709_191501_20170619...
xref_orf_data: SRL_ORF_AXXCNE20170720_083800_20160704...
xref_meteorological_files: SMM_APA_AXVCNE20170619_170611_20170619...
ellipsoid_axis: 6378136.3
ellipsoid_flattening: 0.0033528131778969
<xarray.Dataset>
Dimensions: (meas_ind: 40, time: 2779, wvf_ind: 128)
Coordinates:
* time (time) datetime64[ns] 2017-06-19T15:14:39.356848 ... 2017-06-19T16:04:56.808873920
* meas_ind (meas_ind) int8 0 1 2 3 4 ... 36 37 38 39
* wvf_ind (wvf_ind) int8 0 1 2 3 ... 125 126 127
lat (time) float64 ...
lon (time) float64 ...
lon_40hz (time, meas_ind) float64 ...
lat_40hz (time, meas_ind) float64 ...
Data variables:
time_40hz (time, meas_ind) datetime64[ns] ...
surface_type (time) float32 ...
rad_surf_type (time) float32 ...
qual_alt_1hz_range (time) float32 ...
qual_alt_1hz_swh (time) float32 ...
qual_alt_1hz_sig0 (time) float32 ...
qual_alt_1hz_off_nadir_angle_wf (time) float32 ...
qual_inst_corr_1hz_range (time) float32 ...
qual_inst_corr_1hz_swh (time) float32 ...
qual_inst_corr_1hz_sig0 (time) float32 ...
qual_rad_1hz_tb_k (time) float32 ...
qual_rad_1hz_tb_ka (time) float32 ...
alt_state_flag_acq_mode_40hz (time, meas_ind) float32 ...
alt_state_flag_tracking_mode_40hz (time, meas_ind) float32 ...
orb_state_flag_diode (time) float32 ...
orb_state_flag_rest (time) float32 ...
ecmwf_meteo_map_avail (time) float32 ...
trailing_edge_variation_flag (time) float32 ...
trailing_edge_variation_flag_40hz (time, meas_ind) float32 ...
ice_flag (time) float32 ...
interp_flag_mean_sea_surface (time) float32 ...
interp_flag_mdt (time) float32 ...
interp_flag_ocean_tide_sol1 (time) float32 ...
interp_flag_ocean_tide_sol2 (time) float32 ...
interp_flag_meteo (time) float32 ...
alt (time) float64 ...
alt_40hz (time, meas_ind) float64 ...
orb_alt_rate (time) float32 ...
range (time) float64 ...
range_40hz (time, meas_ind) float64 ...
range_used_40hz (time, meas_ind) float32 ...
range_rms (time) float32 ...
range_numval (time) float32 ...
number_of_iterations (time, meas_ind) float32 ...
net_instr_corr_range (time) float64 ...
model_dry_tropo_corr (time) float32 ...
model_wet_tropo_corr (time) float32 ...
rad_wet_tropo_corr (time) float32 ...
iono_corr_gim (time) float32 ...
sea_state_bias (time) float32 ...
swh (time) float32 ...
swh_40hz (time, meas_ind) float32 ...
swh_used_40hz (time, meas_ind) float32 ...
swh_rms (time) float32 ...
swh_numval (time) float32 ...
net_instr_corr_swh (time) float32 ...
sig0 (time) float32 ...
sig0_40hz (time, meas_ind) float32 ...
sig0_used_40hz (time, meas_ind) float32 ...
sig0_rms (time) float32 ...
sig0_numval (time) float32 ...
agc (time) float32 ...
agc_rms (time) float32 ...
agc_numval (time) float32 ...
net_instr_corr_sig0 (time) float32 ...
atmos_corr_sig0 (time) float32 ...
off_nadir_angle_wf (time) float32 ...
off_nadir_angle_wf_40hz (time, meas_ind) float32 ...
tb_k (time) float32 ...
tb_ka (time) float32 ...
mean_sea_surface (time) float64 ...
mean_topography (time) float64 ...
geoid (time) float64 ...
bathymetry (time) float64 ...
inv_bar_corr (time) float32 ...
hf_fluctuations_corr (time) float32 ...
ocean_tide_sol1 (time) float64 ...
ocean_tide_sol2 (time) float64 ...
ocean_tide_equil (time) float32 ...
ocean_tide_non_equil (time) float32 ...
load_tide_sol1 (time) float32 ...
load_tide_sol2 (time) float32 ...
solid_earth_tide (time) float32 ...
pole_tide (time) float32 ...
wind_speed_model_u (time) float32 ...
wind_speed_model_v (time) float32 ...
wind_speed_alt (time) float32 ...
rad_water_vapor (time) float32 ...
rad_liquid_water (time) float32 ...
ice1_range_40hz (time, meas_ind) float64 ...
ice1_sig0_40hz (time, meas_ind) float32 ...
ice1_qual_flag_40hz (time, meas_ind) float32 ...
seaice_range_40hz (time, meas_ind) float64 ...
seaice_sig0_40hz (time, meas_ind) float32 ...
seaice_qual_flag_40hz (time, meas_ind) float32 ...
ice2_range_40hz (time, meas_ind) float64 ...
ice2_le_sig0_40hz (time, meas_ind) float32 ...
ice2_sig0_40hz (time, meas_ind) float32 ...
ice2_sigmal_40hz (time, meas_ind) float32 ...
ice2_slope1_40hz (time, meas_ind) float64 ...
ice2_slope2_40hz (time, meas_ind) float64 ...
ice2_mqe_40hz (time, meas_ind) float32 ...
ice2_qual_flag_40hz (time, meas_ind) float32 ...
mqe_40hz (time, meas_ind) float32 ...
peakiness_40hz (time, meas_ind) float32 ...
ssha (time) float32 ...
tracker_40hz (time, meas_ind) float64 ...
tracker_used_40hz (time, meas_ind) float32 ...
tracker_diode_40hz (time, meas_ind) float64 ...
pri_counter_40hz (time, meas_ind) float64 ...
qual_alt_1hz_off_nadir_angle_pf (time) float32 ...
off_nadir_angle_pf (time) float32 ...
off_nadir_angle_rain_40hz (time, meas_ind) float32 ...
uso_corr (time) float64 ...
internal_path_delay_corr (time) float64 ...
modeled_instr_corr_range (time) float32 ...
doppler_corr (time) float32 ...
cog_corr (time) float32 ...
modeled_instr_corr_swh (time) float32 ...
internal_corr_sig0 (time) float32 ...
modeled_instr_corr_sig0 (time) float32 ...
agc_40hz (time, meas_ind) float32 ...
agc_corr_40hz (time, meas_ind) float32 ...
scaling_factor_40hz (time, meas_ind) float64 ...
epoch_40hz (time, meas_ind) float64 ...
width_leading_edge_40hz (time, meas_ind) float64 ...
amplitude_40hz (time, meas_ind) float64 ...
thermal_noise_40hz (time, meas_ind) float64 ...
seaice_epoch_40hz (time, meas_ind) float64 ...
seaice_amplitude_40hz (time, meas_ind) float64 ...
ice2_epoch_40hz (time, meas_ind) float64 ...
ice2_amplitude_40hz (time, meas_ind) float64 ...
ice2_mean_amplitude_40hz (time, meas_ind) float64 ...
ice2_thermal_noise_40hz (time, meas_ind) float64 ...
ice2_slope_40hz (time, meas_ind) float64 ...
signal_to_noise_ratio (time) float32 ...
waveforms_40hz (time, meas_ind, wvf_ind) float32 ...
Attributes:
Conventions: CF-1.1
title: GDR - Expertise dataset
institution: CNES
source: radar altimeter
history: 2017-07-21 08:25:19 : Creation
contact: CNES aviso@oceanobs.com, EUMETSAT ops@...
references: L1 library=V4.5p1, L2 library=V5.5p2, ...
processing_center: SALP
reference_document: SARAL/ALTIKA Products Handbook, SALP-M...
mission_name: SARAL
altimeter_sensor_name: ALTIKA
radiometer_sensor_name: ALTIKA_RAD
doris_sensor_name: DGXX
cycle_number: 110
absolute_rev_number: 22546
pass_number: 2
absolute_pass_number: 109220
equator_time: 2017-06-19 15:39:46.492000
equator_longitude: 35.21
first_meas_time: 2017-06-19 15:14:39.356848
last_meas_time: 2017-06-19 16:04:56.808874
xref_altimeter_level1: ALK_ALT_1PaS20170619_154722_20170619_1...
xref_radiometer_level1: ALK_RAD_1PaS20170619_154643_20170619_1...
xref_altimeter_characterisation: ALK_CHA_AXVCNE20131115_120000_20100101...
xref_radiometer_characterisation: ALK_CHR_AXVCNE20110207_180000_20110101...
xref_altimeter_ltm: ALK_CAL_AXXCNE20170720_110014_20130102...
xref_doris_uso: SRL_OS1_AXXCNE20170720_083800_20130226...
xref_orbit_data: SRL_VOR_AXVCNE20170720_111700_20170618...
xref_pf_data: SRL_VPF_AXVCNE20170720_111800_20170618...
xref_pole_location: SMM_POL_AXXCNE20170721_071500_19870101...
xref_gim_data: SRL_ION_AXPCNE20170620_074756_20170619...
xref_mog2d_data: SMM_MOG_AXVCNE20170709_191501_20170619...
xref_orf_data: SRL_ORF_AXXCNE20170720_083800_20160704...
xref_meteorological_files: SMM_APA_AXVCNE20170619_170611_20170619...
ellipsoid_axis: 6378136.3
ellipsoid_flattening: 0.0033528131778969```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,372848074