Skip to content

IDEX l1b: fix temperature conversion segmented poly #3631

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lacoak21 merged 4 commits into from
Nov 10, 2026
Merged

IDEX l1b: fix temperature conversion segmented poly #3631

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
9c56774
update temp conversion
lacoak21 Nov 5, 2026
f6ef894
remove test
lacoak21 Nov 5, 2026
dba154f
update incides
lacoak21 Nov 9, 2026
a6bad27
make test more robust. Field two should have some fractional and nega…
lacoak21 Nov 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 39 additions & 33 deletions imap_processing/idex/idex_variable_unpacking_and_eu_conversion.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,33 +1,39 @@
index,mnemonic,var_name,starting_bit,nbits_padding_before,unsigned_nbits,unit,c0,c1,c2,c3,c4,c5,c6,c7,convertAs,packetName
1,current_1v_pol,idx__txhdrprochkch01,4,4,12,mA,0,0.9029253,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
2,current_1p9v_pol,idx__txhdrprochkch01,20,4,12,mA,0,0.9029253,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
3,temperature_1,idx__txhdrprochkch23,4,4,12,C,162.8276,-0.2668687,0.0003323794,-0.000000243153,9.034532e-11,-1.327735E-14,0,0,UNSEGMENTED_POLY,IDEX_SCI
4,temperature_2,idx__txhdrprochkch23,20,4,12,C,162.8276,-0.2668687,0.0003323794,-0.000000243153,9.034532e-11,-1.327735E-14,0,0,UNSEGMENTED_POLY,IDEX_SCI
5,voltage_1v_bus,idx__txhdrprochkch45,4,4,12,V,0,0.000805861,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
6,fpga_temperature,idx__txhdrprochkch45,20,4,12,C,358.31,-0.1187104,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
7,voltage_1p9v_bus,idx__txhdrprochkch67,4,4,12,V,0,0.000805861,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
8,voltage_3p3v_bus,idx__txhdrprochkch67,20,4,12,V,0,0.001611722,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
9,detector_voltage,idx__txhdrhvpshkch01,4,4,12,V,0,1.4652,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
10,sensor_voltage,idx__txhdrhvpshkch01,20,4,12,V,0,1.4652,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
11,target_voltage,idx__txhdrhvpshkch23,4,4,12,V,0,1.4652,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
12,rejection_voltage,idx__txhdrhvpshkch23,20,4,12,V,0,1.4652,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
13,reflectron_voltage,idx__txhdrhvpshkch45,4,4,12,V,0,1.4652,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
14,current_hvps_sensor,idx__txhdrhvpshkch45,20,4,12,mA,0,0.000007326,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
15,positive_current_hvps,idx__txhdrhvpshkch67,4,4,12,mA,0,0.000024339,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
16,negative_current_hvps,idx__txhdrhvpshkch67,20,4,12,mA,0,0.000024339,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
17,voltage_3p3_ref,idx__txhdrlvhk0ch01,4,4,12,V,0,0.00161172,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
18,voltage_3p3_op_ref,idx__txhdrlvhk0ch01,20,4,12,V,0,0.00161172,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
19,voltage_neg6v_bus,idx__txhdrlvhk0ch23,4,4,12,V,-33,0.00886447,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
20,voltage_pos6v_bus,idx__txhdrlvhk0ch23,20,4,12,V,0,0.00241758,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
21,voltage_pos16v_bus,idx__txhdrlvhk0ch45,4,4,12,V,0,0.00482711,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
22,voltage_pos3p3v_bus,idx__txhdrlvhk0ch45,20,4,12,V,0,0.00161172,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
23,voltage_neg5v_bus,idx__txhdrlvhk0ch67,4,4,12,V,-33,0.00886447,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
24,voltage_pos5v_bus,idx__txhdrlvhk0ch67,20,4,12,V,0,0.00161172,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
25,current_3p3v_bus,idx__txhdrlvhk1ch01,4,4,12,A,0,0.00161172,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
26,current_16v_bus,idx__txhdrlvhk1ch01,20,4,12,A,0,0.000161172,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
27,current_6v_bus,idx__txhdrlvhk1ch23,4,4,12,A,0,0.0000805861,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
28,current_neg6v_bus,idx__txhdrlvhk1ch23,20,4,12,A,0,-0.0000805861,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
29,current_5v_bus,idx__txhdrlvhk1ch45,4,4,12,A,0,0.00004029304,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
30,current_neg5v_bus,idx__txhdrlvhk1ch45,20,4,12,A,0,-0.0000201465,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
31,current_2p5v_bus,idx__txhdrlvhk1ch67,4,4,12,A,0,0.00161172,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
32,current_neg2p5v_bus,idx__txhdrlvhk1ch67,20,4,12,A,0,-0.000040293,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
index,mnemonic,var_name,starting_bit,nbits_padding_before,unsigned_nbits,unit,dn_range_start,dn_range_stop,c0,c1,c2,c3,c4,c5,c6,c7,convertAs,packetName
1,current_1v_pol,idx__txhdrprochkch01,4,4,12,mA,,,0,0.9029253,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
2,current_1p9v_pol,idx__txhdrprochkch01,20,4,12,mA,,,0,0.9029253,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
3,temperature_1,idx__txhdrprochkch23,4,4,12,C,0,2063,162.8276,-0.2668687,0.0003323794,-2.43153e-07,9.034532e-11,-1.327735e-14,0,0,SEGMENTED_POLY,IDEX_SCI
4,temperature_1,idx__txhdrprochkch23,4,4,12,C,2064,3853,714.2124,-1.194011,0.0008599094,-3.145266e-07,5.745416e-11,-4.208231e-15,0,0,SEGMENTED_POLY,IDEX_SCI
5,temperature_1,idx__txhdrprochkch23,4,4,12,C,3854,4051,136881.2,-105.4644,0.02710281,-2.323543e-06,0,0,0,0,SEGMENTED_POLY,IDEX_SCI
6,temperature_1,idx__txhdrprochkch23,4,4,12,C,4052,4095,-100809.9,50.03809,-0.00621209,0,0,0,0,0,SEGMENTED_POLY,IDEX_SCI
7,temperature_2,idx__txhdrprochkch23,20,4,12,C,0,2063,162.8276,-0.2668687,0.0003323794,-2.43153e-07,9.034532e-11,-1.327735e-14,0,0,SEGMENTED_POLY,IDEX_SCI
8,temperature_2,idx__txhdrprochkch23,20,4,12,C,2064,3853,714.2124,-1.194011,0.0008599094,-3.145266e-07,5.745416e-11,-4.208231e-15,0,0,SEGMENTED_POLY,IDEX_SCI
9,temperature_2,idx__txhdrprochkch23,20,4,12,C,3854,4051,136881.2,-105.4644,0.02710281,-2.323543e-06,0,0,0,0,SEGMENTED_POLY,IDEX_SCI
10,temperature_2,idx__txhdrprochkch23,20,4,12,C,4052,4095,-100809.9,50.03809,-0.00621209,0,0,0,0,0,SEGMENTED_POLY,IDEX_SCI
11,voltage_1v_bus,idx__txhdrprochkch45,4,4,12,V,,,0,0.000805861,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
12,fpga_temperature,idx__txhdrprochkch45,20,4,12,C,,,358.31,-0.1187104,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
13,voltage_1p9v_bus,idx__txhdrprochkch67,4,4,12,V,,,0,0.000805861,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
14,voltage_3p3v_bus,idx__txhdrprochkch67,20,4,12,V,,,0,0.001611722,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
15,detector_voltage,idx__txhdrhvpshkch01,4,4,12,V,,,0,1.4652,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
16,sensor_voltage,idx__txhdrhvpshkch01,20,4,12,V,,,0,1.4652,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
17,target_voltage,idx__txhdrhvpshkch23,4,4,12,V,,,0,1.4652,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
18,rejection_voltage,idx__txhdrhvpshkch23,20,4,12,V,,,0,1.4652,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
19,reflectron_voltage,idx__txhdrhvpshkch45,4,4,12,V,,,0,1.4652,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
20,current_hvps_sensor,idx__txhdrhvpshkch45,20,4,12,mA,,,0,0.000007326,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
21,positive_current_hvps,idx__txhdrhvpshkch67,4,4,12,mA,,,0,0.000024339,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
22,negative_current_hvps,idx__txhdrhvpshkch67,20,4,12,mA,,,0,0.000024339,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
23,voltage_3p3_ref,idx__txhdrlvhk0ch01,4,4,12,V,,,0,0.00161172,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
24,voltage_3p3_op_ref,idx__txhdrlvhk0ch01,20,4,12,V,,,0,0.00161172,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
25,voltage_neg6v_bus,idx__txhdrlvhk0ch23,4,4,12,V,,,-33,0.00886447,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
26,voltage_pos6v_bus,idx__txhdrlvhk0ch23,20,4,12,V,,,0,0.00241758,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
27,voltage_pos16v_bus,idx__txhdrlvhk0ch45,4,4,12,V,,,0,0.00482711,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
28,voltage_pos3p3v_bus,idx__txhdrlvhk0ch45,20,4,12,V,,,0,0.00161172,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
29,voltage_neg5v_bus,idx__txhdrlvhk0ch67,4,4,12,V,,,-33,0.00886447,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
30,voltage_pos5v_bus,idx__txhdrlvhk0ch67,20,4,12,V,,,0,0.00161172,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
31,current_3p3v_bus,idx__txhdrlvhk1ch01,4,4,12,A,,,0,0.00161172,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
32,current_16v_bus,idx__txhdrlvhk1ch01,20,4,12,A,,,0,0.000161172,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
33,current_6v_bus,idx__txhdrlvhk1ch23,4,4,12,A,,,0,0.0000805861,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
34,current_neg6v_bus,idx__txhdrlvhk1ch23,20,4,12,A,,,0,-0.0000805861,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
35,current_5v_bus,idx__txhdrlvhk1ch45,4,4,12,A,,,0,0.00004029304,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
36,current_neg5v_bus,idx__txhdrlvhk1ch45,20,4,12,A,,,0,-0.0000201465,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
37,current_2p5v_bus,idx__txhdrlvhk1ch67,4,4,12,A,,,0,0.00161172,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
38,current_neg2p5v_bus,idx__txhdrlvhk1ch67,20,4,12,A,,,0,-0.000040293,0,0,0,0,0,0,UNSEGMENTED_POLY,IDEX_SCI
22 changes: 12 additions & 10 deletions imap_processing/tests/test_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,13 @@ def test_convert_raw_to_eu(tmp_path):
],
"dn_range_start": [0, 0, 1, dn_cutoff + 1, 0],
"dn_range_stop": [0, 0, dn_cutoff, 4, 0],
"c0": [0, 1, 3, 4, 2],
"c1": [0, 1, 3, 4, 2],
"c2": [0, 0, 3, 4, 2],
"c3": [0, 0, 3, 4, 2],
"c5": [0, 1, 3, 4, 2],
"c6": [0, 0, 3, 4, 2],
"c7": [0, 0, 3, 4, 2],
"c0": [0, 1, -3.5, 1.5, 2],
"c1": [0, 1, 0.25, -0.75, 2],
"c2": [0, 0, 0, 4, 2],
"c3": [0, 0, 0, 0, 2],
"c5": [0, 1, 0, 0, 2],
"c6": [0, 0, 0, 0, 2],
"c7": [0, 0, 0, 0, 2],
"unit": ["a", "b", "d", "d", "c"],
}
)
Expand All @@ -49,9 +49,11 @@ def test_convert_raw_to_eu(tmp_path):

# Generate a fake dataset for testing
n_packets = 3
field_0 = np.arange(n_packets)
field_1 = np.arange(n_packets) + 10
field_2 = np.arange(n_packets) + 1
field_0 = np.arange(n_packets).astype(
np.uint16
) # Use unsigned integer to mock packet data.
field_1 = np.arange(n_packets).astype(np.uint16) + 10
field_2 = np.arange(n_packets).astype(np.uint16) + 1
time = np.arange(n_packets) + 1000
dn_dataset = xr.Dataset(
data_vars=dict(
Expand Down
4 changes: 3 additions & 1 deletion imap_processing/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,9 @@ def convert_raw_to_eu(
dataset[var].data = np.polyval(coeff_values, dataset[var].data)

elif np.all(packet_df["convertAs"] == "SEGMENTED_POLY"):
data = dataset[var].data
# np.piecewise output takes the input dtype, so cast to float to avoid
Comment thread
lacoak21 marked this conversation as resolved.
Resolved
# truncating (or wrapping negative) values for integer raw DNs
data = dataset[var].data.astype(np.float64)
Comment thread
lacoak21 marked this conversation as resolved.
Resolved
# Check if any of the raw DN values fall outside the ranges
bad_mask = np.logical_or(
data < packet_df["dn_range_start"].min(),
Expand Down
Loading