html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue https://github.com/pydata/xarray/issues/2501#issuecomment-432546977,https://api.github.com/repos/pydata/xarray/issues/2501,432546977,MDEyOklzc3VlQ29tbWVudDQzMjU0Njk3Nw==,1492047,2018-10-24T07:38:31Z,2018-10-24T07:38:31Z,CONTRIBUTOR,"Thank you for looking into this. I just want to point out that I'm not that much concerned with the ""slow performance"" but much more with the memory consumption and the limitation it implies. ```python from glob import glob import xarray as xr all_files = glob('...*TP110*.nc') display(xr.open_dataset(all_files[0])) display(xr.open_dataset(all_files[1])) ``` ``` Dimensions: (meas_ind: 40, time: 2871, wvf_ind: 128) Coordinates: * time (time) datetime64[ns] 2017-06-19T14:24:20.792036992 ... 2017-06-19T15:14:38.491743104 * meas_ind (meas_ind) int8 0 1 2 3 4 ... 36 37 38 39 * wvf_ind (wvf_ind) int8 0 1 2 3 ... 125 126 127 lat (time) float64 ... lon (time) float64 ... lon_40hz (time, meas_ind) float64 ... lat_40hz (time, meas_ind) float64 ... Data variables: time_40hz (time, meas_ind) datetime64[ns] ... surface_type (time) float32 ... rad_surf_type (time) float32 ... qual_alt_1hz_range (time) float32 ... qual_alt_1hz_swh (time) float32 ... qual_alt_1hz_sig0 (time) float32 ... qual_alt_1hz_off_nadir_angle_wf (time) float32 ... qual_inst_corr_1hz_range (time) float32 ... qual_inst_corr_1hz_swh (time) float32 ... qual_inst_corr_1hz_sig0 (time) float32 ... qual_rad_1hz_tb_k (time) float32 ... qual_rad_1hz_tb_ka (time) float32 ... alt_state_flag_acq_mode_40hz (time, meas_ind) float32 ... alt_state_flag_tracking_mode_40hz (time, meas_ind) float32 ... orb_state_flag_diode (time) float32 ... orb_state_flag_rest (time) float32 ... ecmwf_meteo_map_avail (time) float32 ... trailing_edge_variation_flag (time) float32 ... trailing_edge_variation_flag_40hz (time, meas_ind) float32 ... ice_flag (time) float32 ... interp_flag_mean_sea_surface (time) float32 ... interp_flag_mdt (time) float32 ... interp_flag_ocean_tide_sol1 (time) float32 ... interp_flag_ocean_tide_sol2 (time) float32 ... interp_flag_meteo (time) float32 ... alt (time) float64 ... alt_40hz (time, meas_ind) float64 ... orb_alt_rate (time) float32 ... range (time) float64 ... range_40hz (time, meas_ind) float64 ... range_used_40hz (time, meas_ind) float32 ... range_rms (time) float32 ... range_numval (time) float32 ... number_of_iterations (time, meas_ind) float32 ... net_instr_corr_range (time) float64 ... model_dry_tropo_corr (time) float32 ... model_wet_tropo_corr (time) float32 ... rad_wet_tropo_corr (time) float32 ... iono_corr_gim (time) float32 ... sea_state_bias (time) float32 ... swh (time) float32 ... swh_40hz (time, meas_ind) float32 ... swh_used_40hz (time, meas_ind) float32 ... swh_rms (time) float32 ... swh_numval (time) float32 ... net_instr_corr_swh (time) float32 ... sig0 (time) float32 ... sig0_40hz (time, meas_ind) float32 ... sig0_used_40hz (time, meas_ind) float32 ... sig0_rms (time) float32 ... sig0_numval (time) float32 ... agc (time) float32 ... agc_rms (time) float32 ... agc_numval (time) float32 ... net_instr_corr_sig0 (time) float32 ... atmos_corr_sig0 (time) float32 ... off_nadir_angle_wf (time) float32 ... off_nadir_angle_wf_40hz (time, meas_ind) float32 ... tb_k (time) float32 ... tb_ka (time) float32 ... mean_sea_surface (time) float64 ... mean_topography (time) float64 ... geoid (time) float64 ... bathymetry (time) float64 ... inv_bar_corr (time) float32 ... hf_fluctuations_corr (time) float32 ... ocean_tide_sol1 (time) float64 ... ocean_tide_sol2 (time) float64 ... ocean_tide_equil (time) float32 ... ocean_tide_non_equil (time) float32 ... load_tide_sol1 (time) float32 ... load_tide_sol2 (time) float32 ... solid_earth_tide (time) float32 ... pole_tide (time) float32 ... wind_speed_model_u (time) float32 ... wind_speed_model_v (time) float32 ... wind_speed_alt (time) float32 ... rad_water_vapor (time) float32 ... rad_liquid_water (time) float32 ... ice1_range_40hz (time, meas_ind) float64 ... ice1_sig0_40hz (time, meas_ind) float32 ... ice1_qual_flag_40hz (time, meas_ind) float32 ... seaice_range_40hz (time, meas_ind) float64 ... seaice_sig0_40hz (time, meas_ind) float32 ... seaice_qual_flag_40hz (time, meas_ind) float32 ... ice2_range_40hz (time, meas_ind) float64 ... ice2_le_sig0_40hz (time, meas_ind) float32 ... ice2_sig0_40hz (time, meas_ind) float32 ... ice2_sigmal_40hz (time, meas_ind) float32 ... ice2_slope1_40hz (time, meas_ind) float64 ... ice2_slope2_40hz (time, meas_ind) float64 ... ice2_mqe_40hz (time, meas_ind) float32 ... ice2_qual_flag_40hz (time, meas_ind) float32 ... mqe_40hz (time, meas_ind) float32 ... peakiness_40hz (time, meas_ind) float32 ... ssha (time) float32 ... tracker_40hz (time, meas_ind) float64 ... tracker_used_40hz (time, meas_ind) float32 ... tracker_diode_40hz (time, meas_ind) float64 ... pri_counter_40hz (time, meas_ind) float64 ... qual_alt_1hz_off_nadir_angle_pf (time) float32 ... off_nadir_angle_pf (time) float32 ... off_nadir_angle_rain_40hz (time, meas_ind) float32 ... uso_corr (time) float64 ... internal_path_delay_corr (time) float64 ... modeled_instr_corr_range (time) float32 ... doppler_corr (time) float32 ... cog_corr (time) float32 ... modeled_instr_corr_swh (time) float32 ... internal_corr_sig0 (time) float32 ... modeled_instr_corr_sig0 (time) float32 ... agc_40hz (time, meas_ind) float32 ... agc_corr_40hz (time, meas_ind) float32 ... scaling_factor_40hz (time, meas_ind) float64 ... epoch_40hz (time, meas_ind) float64 ... width_leading_edge_40hz (time, meas_ind) float64 ... amplitude_40hz (time, meas_ind) float64 ... thermal_noise_40hz (time, meas_ind) float64 ... seaice_epoch_40hz (time, meas_ind) float64 ... seaice_amplitude_40hz (time, meas_ind) float64 ... ice2_epoch_40hz (time, meas_ind) float64 ... ice2_amplitude_40hz (time, meas_ind) float64 ... ice2_mean_amplitude_40hz (time, meas_ind) float64 ... ice2_thermal_noise_40hz (time, meas_ind) float64 ... ice2_slope_40hz (time, meas_ind) float64 ... signal_to_noise_ratio (time) float32 ... waveforms_40hz (time, meas_ind, wvf_ind) float32 ... Attributes: Conventions: CF-1.1 title: GDR - Expertise dataset institution: CNES source: radar altimeter history: 2017-07-21 08:25:07 : Creation contact: CNES aviso@oceanobs.com, EUMETSAT ops@... references: L1 library=V4.5p1, L2 library=V5.5p2, ... processing_center: SALP reference_document: SARAL/ALTIKA Products Handbook, SALP-M... mission_name: SARAL altimeter_sensor_name: ALTIKA radiometer_sensor_name: ALTIKA_RAD doris_sensor_name: DGXX cycle_number: 110 absolute_rev_number: 22545 pass_number: 1 absolute_pass_number: 109219 equator_time: 2017-06-19 14:49:32.128000 equator_longitude: 227.77 first_meas_time: 2017-06-19 14:24:20.792037 last_meas_time: 2017-06-19 15:14:38.491743 xref_altimeter_level1: ALK_ALT_1PaS20170619_154722_20170619_1... xref_radiometer_level1: ALK_RAD_1PaS20170619_154643_20170619_1... xref_altimeter_characterisation: ALK_CHA_AXVCNE20131115_120000_20100101... xref_radiometer_characterisation: ALK_CHR_AXVCNE20110207_180000_20110101... xref_altimeter_ltm: ALK_CAL_AXXCNE20170720_110014_20130102... xref_doris_uso: SRL_OS1_AXXCNE20170720_083800_20130226... xref_orbit_data: SRL_VOR_AXVCNE20170720_111700_20170618... xref_pf_data: SRL_VPF_AXVCNE20170720_111800_20170618... xref_pole_location: SMM_POL_AXXCNE20170721_071500_19870101... xref_gim_data: SRL_ION_AXPCNE20170620_074756_20170619... xref_mog2d_data: SMM_MOG_AXVCNE20170709_191501_20170619... xref_orf_data: SRL_ORF_AXXCNE20170720_083800_20160704... xref_meteorological_files: SMM_APA_AXVCNE20170619_170611_20170619... ellipsoid_axis: 6378136.3 ellipsoid_flattening: 0.0033528131778969 Dimensions: (meas_ind: 40, time: 2779, wvf_ind: 128) Coordinates: * time (time) datetime64[ns] 2017-06-19T15:14:39.356848 ... 2017-06-19T16:04:56.808873920 * meas_ind (meas_ind) int8 0 1 2 3 4 ... 36 37 38 39 * wvf_ind (wvf_ind) int8 0 1 2 3 ... 125 126 127 lat (time) float64 ... lon (time) float64 ... lon_40hz (time, meas_ind) float64 ... lat_40hz (time, meas_ind) float64 ... Data variables: time_40hz (time, meas_ind) datetime64[ns] ... surface_type (time) float32 ... rad_surf_type (time) float32 ... qual_alt_1hz_range (time) float32 ... qual_alt_1hz_swh (time) float32 ... qual_alt_1hz_sig0 (time) float32 ... qual_alt_1hz_off_nadir_angle_wf (time) float32 ... qual_inst_corr_1hz_range (time) float32 ... qual_inst_corr_1hz_swh (time) float32 ... qual_inst_corr_1hz_sig0 (time) float32 ... qual_rad_1hz_tb_k (time) float32 ... qual_rad_1hz_tb_ka (time) float32 ... alt_state_flag_acq_mode_40hz (time, meas_ind) float32 ... alt_state_flag_tracking_mode_40hz (time, meas_ind) float32 ... orb_state_flag_diode (time) float32 ... orb_state_flag_rest (time) float32 ... ecmwf_meteo_map_avail (time) float32 ... trailing_edge_variation_flag (time) float32 ... trailing_edge_variation_flag_40hz (time, meas_ind) float32 ... ice_flag (time) float32 ... interp_flag_mean_sea_surface (time) float32 ... interp_flag_mdt (time) float32 ... interp_flag_ocean_tide_sol1 (time) float32 ... interp_flag_ocean_tide_sol2 (time) float32 ... interp_flag_meteo (time) float32 ... alt (time) float64 ... alt_40hz (time, meas_ind) float64 ... orb_alt_rate (time) float32 ... range (time) float64 ... range_40hz (time, meas_ind) float64 ... range_used_40hz (time, meas_ind) float32 ... range_rms (time) float32 ... range_numval (time) float32 ... number_of_iterations (time, meas_ind) float32 ... net_instr_corr_range (time) float64 ... model_dry_tropo_corr (time) float32 ... model_wet_tropo_corr (time) float32 ... rad_wet_tropo_corr (time) float32 ... iono_corr_gim (time) float32 ... sea_state_bias (time) float32 ... swh (time) float32 ... swh_40hz (time, meas_ind) float32 ... swh_used_40hz (time, meas_ind) float32 ... swh_rms (time) float32 ... swh_numval (time) float32 ... net_instr_corr_swh (time) float32 ... sig0 (time) float32 ... sig0_40hz (time, meas_ind) float32 ... sig0_used_40hz (time, meas_ind) float32 ... sig0_rms (time) float32 ... sig0_numval (time) float32 ... agc (time) float32 ... agc_rms (time) float32 ... agc_numval (time) float32 ... net_instr_corr_sig0 (time) float32 ... atmos_corr_sig0 (time) float32 ... off_nadir_angle_wf (time) float32 ... off_nadir_angle_wf_40hz (time, meas_ind) float32 ... tb_k (time) float32 ... tb_ka (time) float32 ... mean_sea_surface (time) float64 ... mean_topography (time) float64 ... geoid (time) float64 ... bathymetry (time) float64 ... inv_bar_corr (time) float32 ... hf_fluctuations_corr (time) float32 ... ocean_tide_sol1 (time) float64 ... ocean_tide_sol2 (time) float64 ... ocean_tide_equil (time) float32 ... ocean_tide_non_equil (time) float32 ... load_tide_sol1 (time) float32 ... load_tide_sol2 (time) float32 ... solid_earth_tide (time) float32 ... pole_tide (time) float32 ... wind_speed_model_u (time) float32 ... wind_speed_model_v (time) float32 ... wind_speed_alt (time) float32 ... rad_water_vapor (time) float32 ... rad_liquid_water (time) float32 ... ice1_range_40hz (time, meas_ind) float64 ... ice1_sig0_40hz (time, meas_ind) float32 ... ice1_qual_flag_40hz (time, meas_ind) float32 ... seaice_range_40hz (time, meas_ind) float64 ... seaice_sig0_40hz (time, meas_ind) float32 ... seaice_qual_flag_40hz (time, meas_ind) float32 ... ice2_range_40hz (time, meas_ind) float64 ... ice2_le_sig0_40hz (time, meas_ind) float32 ... ice2_sig0_40hz (time, meas_ind) float32 ... ice2_sigmal_40hz (time, meas_ind) float32 ... ice2_slope1_40hz (time, meas_ind) float64 ... ice2_slope2_40hz (time, meas_ind) float64 ... ice2_mqe_40hz (time, meas_ind) float32 ... ice2_qual_flag_40hz (time, meas_ind) float32 ... mqe_40hz (time, meas_ind) float32 ... peakiness_40hz (time, meas_ind) float32 ... ssha (time) float32 ... tracker_40hz (time, meas_ind) float64 ... tracker_used_40hz (time, meas_ind) float32 ... tracker_diode_40hz (time, meas_ind) float64 ... pri_counter_40hz (time, meas_ind) float64 ... qual_alt_1hz_off_nadir_angle_pf (time) float32 ... off_nadir_angle_pf (time) float32 ... off_nadir_angle_rain_40hz (time, meas_ind) float32 ... uso_corr (time) float64 ... internal_path_delay_corr (time) float64 ... modeled_instr_corr_range (time) float32 ... doppler_corr (time) float32 ... cog_corr (time) float32 ... modeled_instr_corr_swh (time) float32 ... internal_corr_sig0 (time) float32 ... modeled_instr_corr_sig0 (time) float32 ... agc_40hz (time, meas_ind) float32 ... agc_corr_40hz (time, meas_ind) float32 ... scaling_factor_40hz (time, meas_ind) float64 ... epoch_40hz (time, meas_ind) float64 ... width_leading_edge_40hz (time, meas_ind) float64 ... amplitude_40hz (time, meas_ind) float64 ... thermal_noise_40hz (time, meas_ind) float64 ... seaice_epoch_40hz (time, meas_ind) float64 ... seaice_amplitude_40hz (time, meas_ind) float64 ... ice2_epoch_40hz (time, meas_ind) float64 ... ice2_amplitude_40hz (time, meas_ind) float64 ... ice2_mean_amplitude_40hz (time, meas_ind) float64 ... ice2_thermal_noise_40hz (time, meas_ind) float64 ... ice2_slope_40hz (time, meas_ind) float64 ... signal_to_noise_ratio (time) float32 ... waveforms_40hz (time, meas_ind, wvf_ind) float32 ... Attributes: Conventions: CF-1.1 title: GDR - Expertise dataset institution: CNES source: radar altimeter history: 2017-07-21 08:25:19 : Creation contact: CNES aviso@oceanobs.com, EUMETSAT ops@... references: L1 library=V4.5p1, L2 library=V5.5p2, ... processing_center: SALP reference_document: SARAL/ALTIKA Products Handbook, SALP-M... mission_name: SARAL altimeter_sensor_name: ALTIKA radiometer_sensor_name: ALTIKA_RAD doris_sensor_name: DGXX cycle_number: 110 absolute_rev_number: 22546 pass_number: 2 absolute_pass_number: 109220 equator_time: 2017-06-19 15:39:46.492000 equator_longitude: 35.21 first_meas_time: 2017-06-19 15:14:39.356848 last_meas_time: 2017-06-19 16:04:56.808874 xref_altimeter_level1: ALK_ALT_1PaS20170619_154722_20170619_1... xref_radiometer_level1: ALK_RAD_1PaS20170619_154643_20170619_1... xref_altimeter_characterisation: ALK_CHA_AXVCNE20131115_120000_20100101... xref_radiometer_characterisation: ALK_CHR_AXVCNE20110207_180000_20110101... xref_altimeter_ltm: ALK_CAL_AXXCNE20170720_110014_20130102... xref_doris_uso: SRL_OS1_AXXCNE20170720_083800_20130226... xref_orbit_data: SRL_VOR_AXVCNE20170720_111700_20170618... xref_pf_data: SRL_VPF_AXVCNE20170720_111800_20170618... xref_pole_location: SMM_POL_AXXCNE20170721_071500_19870101... xref_gim_data: SRL_ION_AXPCNE20170620_074756_20170619... xref_mog2d_data: SMM_MOG_AXVCNE20170709_191501_20170619... xref_orf_data: SRL_ORF_AXXCNE20170720_083800_20160704... xref_meteorological_files: SMM_APA_AXVCNE20170619_170611_20170619... ellipsoid_axis: 6378136.3 ellipsoid_flattening: 0.0033528131778969```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,372848074