diff --git a/crates/bashkit/docs/threat-model.md b/crates/bashkit/docs/threat-model.md index 152abca7b..6b92ed271 100644 --- a/crates/bashkit/docs/threat-model.md +++ b/crates/bashkit/docs/threat-model.md @@ -128,7 +128,7 @@ through configurable limits. | Unbounded builtin output (TM-DOS-058) | `seq 1 1000000` produces 1M lines | Add `max_stdout_bytes` limit | **MITIGATED** | | Silent truncation at builtin caps (TM-DOS-109) | `seq 200000`, an awk loop past its cap, or an oversized `sprintf` expression returns incomplete output with exit 0 | Caps report `: limit () exceeded` on stderr and exit non-zero; awk caps and formatting errors are fatal | **MITIGATED** | | In-builtin memory growth (TM-DOS-110) | `awk 'BEGIN { s = "x"; while (1) s = s s }'` or `jq -n '"x" \| until(false; . + .)'` allocates until the host aborts | awk checks each string against a 16 MiB cap before allocating it, caps `$N` field indexes, and caps total variable memory at `max_live_intermediate_bytes` (fatal, exit 2). jq meters every live string, array and object against the same limit and fails before growing (exit 5); non-emitting jq loops and pure recursion stop at the timeout; non-tail recursion hits a live-context ceiling (64) before host stack exhaustion | **MITIGATED** | -| Arrays bypass the retained-memory budget (TM-DOS-114) | Ten array slots each holding a 4 MiB string retain ~40 MB past the 10 MB variable-byte budget | Array keys and values charge `max_total_variable_bytes` alongside scalars, are released on unset/replacement/scope pop, and an over-budget write fails execution | **MITIGATED** | +| Arrays bypass the retained-memory budget (TM-DOS-114) | Ten array slots each holding a 4 MiB string retain ~40 MB past the 10 MB variable-byte budget | Array keys and values charge `max_total_variable_bytes` alongside scalars, are released on unset/replacement/scope pop, and an over-budget write fails execution. Pending indexed compound-assignment keys, values, fields and containers also hold shared intermediate-memory leases, including repeated subscripts | **MITIGATED** | | `$( { let v = self.transform_element(attrs, None, field); @@ -1419,22 +1425,25 @@ impl Interpreter { // subscripts in order against the array as it is being built: // `a=([0]=1+2 [a[0]]=x)` stores x at 3. enum Item { - Keyed(String, String, bool), - Fields(Vec), + Keyed(BudgetedString, BudgetedString, bool), + Fields(BudgetedVec), } - let mut items = Vec::with_capacity(words.len()); + // THREAT[TM-DOS-114]: duplicate indices still retain every expanded + // value until subscripts run; lease both payloads and containers. + let mut items = BudgetedVec::new(Some(&self.execution_budget))?; let mut budget = max_entries.saturating_sub(map.len()); for word in words { + self.execution_budget.consume_work(1)?; // Bash brace-expands indexed elements before spotting // `[i]=`, so `([2]=v{1,2})` stores the plain words // `[2]=v1 [2]=v2`; assoc elements never brace-expand. let braces = self.brace_expand_word(word).is_some(); if let Some((kw, vw, kappend)) = split_keyed_word(word).filter(|_| !braces) { - let key_text = self.expand_word(&kw).await?; + let key_text = self.expand_array_word(&kw).await?; // `[k]=~:~` tilde-expands like an assignment value. let vw = self.tilde_assignment_value(&vw).into_owned(); - let val = self.expand_word(&vw).await?; - items.push(Item::Keyed(key_text, val, kappend)); + let val = self.expand_array_word(&vw).await?; + items.try_push(Item::Keyed(key_text, val, kappend))?; continue; } if budget == 0 { @@ -1442,8 +1451,9 @@ impl Interpreter { } let fields = self.expand_element_fields(word, budget).await?; budget = budget.saturating_sub(fields.len()); - items.push(Item::Fields(fields)); + items.try_push(Item::Fields(fields))?; } + let (items, _items_lease) = items.into_parts(); // Only a subscript that names the array needs it live (keeps // the common case free of per-element copies). let mentions_self = items @@ -1467,9 +1477,10 @@ impl Interpreter { let len = map.keys().max().map_or(0, |m| m + 1) as i64; let i = len + raw; if i < 0 { - return Err(crate::error::Error::LineAbort( - self.diag(format!("{name}[{key_text}]: bad array subscript\n")), - )); + return Err(crate::error::Error::LineAbort(self.diag(format!( + "{name}[{}]: bad array subscript\n", + &*key_text + )))); } i as usize } else { @@ -1480,7 +1491,7 @@ impl Interpreter { } else { None }; - let v = self.transform_element(attrs, old.as_deref(), val); + let v = self.transform_element(attrs, old.as_deref(), val.into_inner()); if map.len() >= max_entries && !map.contains_key(&idx) { break 'items; } @@ -1491,7 +1502,9 @@ impl Interpreter { next = idx + 1; } Item::Fields(fields) => { + let (fields, _fields_lease) = fields.into_parts(); for field in fields { + let field = field.into_inner(); if map.len() >= max_entries { break 'items; } @@ -1516,24 +1529,44 @@ impl Interpreter { /// Expand one unkeyed compound element into fields: word splitting for /// unquoted expansions (bounded by `limit`), `"${a[@]}"` splats, brace /// expansion and globbing for unquoted literals. - async fn expand_element_fields(&mut self, word: &Word, limit: usize) -> Result> { + async fn expand_element_fields( + &mut self, + word: &Word, + limit: usize, + ) -> Result> { // Brace expansion runs first on the unexpanded word, as for command // arguments: `a=("x"{1,2})` gives `x1 x2`, `a=('{a,b}')` stays literal. let Some(braced) = self.brace_expand_word(word) else { - return self.expand_element_word(word, limit).await; + let fields = self.expand_element_word(word, limit).await?; + return self.lease_array_fields(fields); }; - let mut fields = Vec::new(); + let mut fields = BudgetedVec::new(Some(&self.execution_budget))?; for w in &braced { let left = limit.saturating_sub(fields.len()); if left == 0 { break; } - fields.extend(self.expand_element_word(w, left).await?); + let expanded = self.expand_element_word(w, left).await?; + let leased = self.lease_array_fields(expanded)?; + let (expanded, _lease) = leased.into_parts(); + for field in expanded { + fields.try_push(field)?; + } } - fields.truncate(limit); Ok(fields) } + fn lease_array_fields(&self, fields: Vec) -> Result> { + let mut leased = BudgetedVec::new(Some(&self.execution_budget))?; + for field in fields { + leased.try_push(BudgetedString::try_from_string( + field, + Some(&self.execution_budget), + )?)?; + } + Ok(leased) + } + /// One brace-expanded compound element: splitting, splats, then globbing /// for words with unquoted literal text. async fn expand_element_word(&mut self, word: &Word, limit: usize) -> Result> { @@ -1560,7 +1593,7 @@ impl Interpreter { if is_unquoted_expansion { // Split fields are pathname-expanded like command words: // `p='*.txt'; a=($p)` holds the matching files. - let expanded = self.expand_word(word).await?; + let expanded = self.expand_array_word(word).await?; let mut fields = Vec::new(); for field in self.ifs_split_limited(&expanded, limit)? { match self.expand_glob_item(&field, false).await { @@ -1581,7 +1614,7 @@ impl Interpreter { fields.truncate(limit); return Ok(fields); } - let value = self.expand_word(word).await?; + let value = self.expand_array_word(word).await?; let literal_only = word.parts.iter().all(|p| matches!(p, WordPart::Literal(_))); let globbable = if word.quoted { word.has_unquoted_glob @@ -1589,7 +1622,7 @@ impl Interpreter { literal_only }; if !globbable { - return Ok(vec![value]); + return Ok(vec![value.into_inner()]); } let mut fields = match self .expand_glob_item(&value, word.quoted && word.has_unquoted_glob) diff --git a/crates/bashkit/src/interpreter/expansion.rs b/crates/bashkit/src/interpreter/expansion.rs index cfd3720f0..e95595047 100644 --- a/crates/bashkit/src/interpreter/expansion.rs +++ b/crates/bashkit/src/interpreter/expansion.rs @@ -301,11 +301,44 @@ impl Interpreter { } } - pub(super) fn append_expansion_for_word(result: &mut String, word: &Word, value: &str) { + fn append_expansion_for_word( + result: &mut BudgetedString, + word: &Word, + value: &str, + ) -> Result<()> { if word.quoted && word.has_unquoted_glob { - result.push_str(&Self::quote_expansion_for_quoted_glob(value)); + for ch in value.chars() { + if Self::needs_glob_escape(ch) { + result.try_push('\\')?; + } + result.try_push(ch)?; + } } else { - result.push_str(value); + result.try_push_str(value)?; + } + Ok(()) + } + + /// Compound-array values remain live until subscript evaluation. + pub(super) async fn expand_array_word(&mut self, word: &Word) -> Result { + let expanded = Box::pin(self.expand_word_buffer(word, None, true)).await?; + if word.quoted && word.has_unquoted_glob { + let mut unescaped = BudgetedString::new(Some(&self.execution_budget))?; + let mut chars = expanded.chars().peekable(); + while let Some(ch) = chars.next() { + if ch == '\\' + && chars + .peek() + .is_some_and(|next| Self::needs_glob_escape(*next)) + { + unescaped.try_push(chars.next().unwrap_or(ch))?; + } else { + unescaped.try_push(ch)?; + } + } + Ok(unescaped) + } else { + Ok(expanded) } } @@ -329,17 +362,35 @@ impl Interpreter { pub(super) async fn expand_word_inner( &mut self, word: &Word, - mut prompts: Option>, + prompts: Option>, ) -> Result { + // THREAT[TM-DOS-089]: pin the buffer future so the loop state machine + // lives on the heap, not in this frame. Every `$(...)` nesting level + // awaits through here; an inline child future grows per-level stack + // and aborts depth-32 under coverage instrumentation (cf. #1089). + Ok(Box::pin(self.expand_word_buffer(word, prompts, false)) + .await? + .into_inner()) + } + + async fn expand_word_buffer( + &mut self, + word: &Word, + mut prompts: Option>, + leased: bool, + ) -> Result { if prompts.is_none() && (Self::has_indirect_part(word) || Self::has_prompt_part(word)) && let Some(expanded) = self.expand_word_special(word).await? { - return Ok(expanded); - } - let mut result = String::new(); - // Keep command-substitution bytes charged until the complete word has - // been consumed; sibling substitutions otherwise evade live-byte caps. + return Ok(BudgetedString::try_from_string( + expanded, + leased.then_some(&self.execution_budget), + )?); + } + let mut result = BudgetedString::new(leased.then_some(&self.execution_budget))?; + // Unleased words retain substitution charges until consumed. Leased + // words own their buffer charge, so temporary output charges end at copy. let mut substitution_leases = Vec::new(); for (idx, part) in word.parts.iter().enumerate() { @@ -360,7 +411,7 @@ impl Interpreter { raw: None, } .to_string(); - result.push_str(&text); + result.try_push_str(&text)?; } WordPart::Literal(s) => { // Tilde expansion (`~`, `~/x`, `x=~:~`); quoted literals @@ -369,8 +420,8 @@ impl Interpreter { .then(|| self.tilde_expand_word_literal(word, idx, s)) .flatten() { - Some(expanded) => result.push_str(&expanded), - None => result.push_str(s), + Some(expanded) => result.try_push_str(&expanded)?, + None => result.try_push_str(s)?, } } WordPart::Variable(name) => { @@ -391,13 +442,13 @@ impl Interpreter { .unwrap_or_default(), None => " ".to_string(), }; - Self::append_expansion_for_word(&mut result, word, &positional.join(&sep)); + Self::append_expansion_for_word(&mut result, word, &positional.join(&sep))?; } else { Self::append_expansion_for_word( &mut result, word, - &self.expand_variable(name), - ); + &self.expand_variable_cow(name), + )?; } } WordPart::CommandSubstitution(commands) => { @@ -410,9 +461,16 @@ impl Interpreter { // THREAT[TM-DOS-089]: Delegate to Box::pin-ed helper to // prevent stack growth proportional to nesting depth. let trimmed = self.execute_cmd_subst(commands).await?; - let appended_bytes = Self::expansion_appended_len(word, &trimmed); - substitution_leases.push(self.execution_budget.lease_bytes(appended_bytes)?); - Self::append_expansion_for_word(&mut result, word, &trimmed); + let output_bytes = if leased { + trimmed.capacity() + } else { + Self::expansion_appended_len(word, &trimmed) + }; + let output_lease = self.execution_budget.lease_bytes(output_bytes)?; + Self::append_expansion_for_word(&mut result, word, &trimmed)?; + if !leased { + substitution_leases.push(output_lease); + } } WordPart::ArithmeticExpansion(expr) => { let expanded_expr = if expr.contains("$(") || expr.contains('`') { @@ -424,7 +482,7 @@ impl Interpreter { let value = self .try_evaluate_arithmetic_with_assign(&expanded_expr) .map_err(|msg| crate::error::Error::LineAbort(self.arith_diag("", &msg)))?; - Self::append_expansion_for_word(&mut result, word, &value.to_string()); + Self::append_expansion_for_word(&mut result, word, &value.to_string())?; } WordPart::Length(name) => { let value = if let Some(bracket_pos) = name.find('[') { @@ -440,13 +498,13 @@ impl Interpreter { self.expand_array_access_part(arr_name, index_str) } else if name == "@" || name == "*" { // `${#@}` / `${#*}` count the positional parameters. - result.push_str(&self.expand_variable("#")); + result.try_push_str(&self.expand_variable("#"))?; continue; } else { self.nounset_check_plain(name); self.expand_variable(name) }; - result.push_str(&self.shell_length(&value).to_string()); + result.try_push_str(&self.shell_length(&value).to_string())?; } WordPart::ParameterExpansion { name, @@ -524,7 +582,7 @@ impl Interpreter { is_set, ); self.operand_outer_unquoted = false; - Self::append_expansion_for_word(&mut result, word, &expanded); + Self::append_expansion_for_word(&mut result, word, &expanded)?; } WordPart::ArrayAccess { name, index } => { // `${a[$(cmd)]}` in the source: the substitution runs @@ -543,7 +601,7 @@ impl Interpreter { } else { self.expand_array_access_part(name, index) }; - Self::append_expansion_for_word(&mut result, word, &value); + Self::append_expansion_for_word(&mut result, word, &value)?; } WordPart::ArrayIndices { name, star } => { let keys = self.array_keys(name); @@ -552,7 +610,7 @@ impl Interpreter { } else { " ".to_string() }; - Self::append_expansion_for_word(&mut result, word, &keys.join(&sep)); + Self::append_expansion_for_word(&mut result, word, &keys.join(&sep))?; } WordPart::Substring { name, @@ -563,7 +621,7 @@ impl Interpreter { let value = self .substring_part(name, offset, length.as_deref()) .map_err(crate::error::Error::LineAbort)?; - Self::append_expansion_for_word(&mut result, word, &value); + Self::append_expansion_for_word(&mut result, word, &value)?; } WordPart::IndirectExpansion { name, @@ -578,7 +636,7 @@ impl Interpreter { // Nameref without operator: ${!ref} is the name // at the end of the nameref chain. let target = self.resolve_nameref(name).to_string(); - Self::append_expansion_for_word(&mut result, word, &target); + Self::append_expansion_for_word(&mut result, word, &target)?; } else { // Resolve the indirect target variable name let resolved_name = if let Some(target) = nameref_target { @@ -599,16 +657,16 @@ impl Interpreter { *colon_variant, is_set, ); - Self::append_expansion_for_word(&mut result, word, &expanded); + Self::append_expansion_for_word(&mut result, word, &expanded)?; } else { // Plain indirect expansion (no operator) if let Some(arr) = self.scoped.arrays.get(&resolved_name) { if let Some(first) = arr.get(&0) { - Self::append_expansion_for_word(&mut result, word, first); + Self::append_expansion_for_word(&mut result, word, first)?; } } else { let value = self.expand_variable(&resolved_name); - Self::append_expansion_for_word(&mut result, word, &value); + Self::append_expansion_for_word(&mut result, word, &value)?; } } } @@ -626,7 +684,7 @@ impl Interpreter { } else { " ".to_string() }; - Self::append_expansion_for_word(&mut result, word, &names.join(&sep)); + Self::append_expansion_for_word(&mut result, word, &names.join(&sep))?; } WordPart::ArrayLength(name) => { let resolved = self.resolve_nameref(name); @@ -637,20 +695,20 @@ impl Interpreter { } else { usize::from(self.scalar_value(resolved).is_some()) }; - result.push_str(&len.to_string()); + result.try_push_str(&len.to_string())?; } WordPart::ProcessSubstitution { commands, is_input } => { let expanded = self .expand_process_substitution(commands, *is_input) .await?; - Self::append_expansion_for_word(&mut result, word, &expanded); + Self::append_expansion_for_word(&mut result, word, &expanded)?; } WordPart::Transformation { name, operator } => { let value = match prompts.as_mut().and_then(Vec::pop) { Some(decoded) if *operator == 'P' => decoded, _ => self.transformation_part(name, *operator), }; - Self::append_expansion_for_word(&mut result, word, &value); + Self::append_expansion_for_word(&mut result, word, &value)?; } } } @@ -3660,8 +3718,8 @@ mod expansion_charge_tests { for has_unquoted_glob in [false, true] { let w = word(quoted, has_unquoted_glob); for value in values { - let mut appended = String::new(); - Interpreter::append_expansion_for_word(&mut appended, &w, value); + let mut appended = BudgetedString::new(None).unwrap(); + Interpreter::append_expansion_for_word(&mut appended, &w, value).unwrap(); assert_eq!( Interpreter::expansion_appended_len(&w, value), appended.len(), diff --git a/crates/bashkit/src/interpreter/mod.rs b/crates/bashkit/src/interpreter/mod.rs index 87a32e96d..cc41fd5e6 100644 --- a/crates/bashkit/src/interpreter/mod.rs +++ b/crates/bashkit/src/interpreter/mod.rs @@ -14155,22 +14155,31 @@ impl Interpreter { } fn lookup_regular_variable(&self, name: &str) -> Option { + self.lookup_regular_variable_ref(name).map(str::to_owned) + } + + fn lookup_regular_variable_ref(&self, name: &str) -> Option<&str> { if let Some(value) = self.scoped.variables.get(name) { - return Some(value.clone()); + return Some(value.as_str()); } // `$arr` on an array is `${arr[0]}` (indexed) or `${arr["0"]}` (assoc). if let Some(arr) = self.scoped.arrays.get(name) { - return arr.get(&0).cloned(); + return arr.get(&0).map(String::as_str); } if let Some(arr) = self.scoped.assoc_arrays.get(name) { - return arr.get("0").cloned(); + return arr.get("0").map(String::as_str); } - self.env.get(name).cloned() + self.env.get(name).map(String::as_str) } fn expand_variable(&self, name: &str) -> String { + self.expand_variable_cow(name).into_owned() + } + + fn expand_variable_cow(&self, name: &str) -> std::borrow::Cow<'_, str> { + use std::borrow::Cow; // Resolve nameref before expansion let name = self.resolve_nameref(name); @@ -14185,71 +14194,71 @@ impl Interpreter { // Plain variable values only, so a reference cannot recurse. if idx_str.contains(['$', '"', '\'', '\\']) { let key = self.expand_key_text_with(idx_str, true); - return arr.get(&key).cloned().unwrap_or_default(); + return Cow::Owned(arr.get(&key).cloned().unwrap_or_default()); } - return arr.get(idx_str).cloned().unwrap_or_default(); + return Cow::Owned(arr.get(idx_str).cloned().unwrap_or_default()); } else if let Some(arr) = self.scoped.arrays.get(arr_name) { let idx: usize = self.evaluate_arithmetic(idx_str).try_into().unwrap_or(0); - return arr.get(&idx).cloned().unwrap_or_default(); + return Cow::Owned(arr.get(&idx).cloned().unwrap_or_default()); } - return String::new(); + return Cow::Owned(String::new()); } // Check for special parameters (POSIX required) match name { - "?" => return self.last_exit_code.to_string(), + "?" => return Cow::Owned(self.last_exit_code.to_string()), "#" => { // Number of positional parameters if let Some(frame) = self.call_stack.last() { - return frame.positional.len().to_string(); + return Cow::Owned(frame.positional.len().to_string()); } - return "0".to_string(); + return Cow::Owned("0".to_string()); } "@" => { // All positional parameters (space-separated as string) if let Some(frame) = self.call_stack.last() { - return frame.positional.join(" "); + return Cow::Owned(frame.positional.join(" ")); } - return String::new(); + return Cow::Owned(String::new()); } "*" => { // All positional parameters joined by IFS first char if let Some(frame) = self.call_stack.last() { let sep = self.get_ifs_separator(); - return frame.positional.join(&sep); + return Cow::Owned(frame.positional.join(&sep)); } - return String::new(); + return Cow::Owned(String::new()); } // THREAT[TM-INF-014]: Return sandboxed PID, not real host PID. "$" => { - return "1".to_string(); + return Cow::Owned("1".to_string()); } // A subshell's own virtual pid (`$$` stays the shell's). "BASHPID" => { - return if self.bashpid == 0 { + return Cow::Owned(if self.bashpid == 0 { "1".to_string() } else { self.bashpid.to_string() - }; + }); } "!" => { // $! - PID of most recent background command // In Bashkit's virtual environment, background jobs run synchronously // Return empty string or last job ID placeholder if let Some(last_bg_pid) = &self.last_bg_pid { - return last_bg_pid.clone(); + return Cow::Owned(last_bg_pid.clone()); } - return String::new(); + return Cow::Owned(String::new()); } "-" => { // $- - Current option flags, from the SHOPT_* variables. let flags = builtins::dollar_dash(&self.scoped.variables); if !self.interactive { - return flags; + return Cow::Owned(flags); } // `bash -i`: `i` follows `h` (`himBHc` order). let at = flags.find('h').map_or(0, |i| i + 1); - return format!("{}i{}", &flags[..at], &flags[at..]); + return Cow::Owned(format!("{}i{}", &flags[..at], &flags[at..])); } "RANDOM" => { // $RANDOM - LCG matching bash behavior, seeded per-instance. @@ -14257,60 +14266,61 @@ impl Interpreter { let prev = self.random_state.load(Ordering::Relaxed); let next = prev.wrapping_mul(1103515245).wrapping_add(12345); self.random_state.store(next, Ordering::Relaxed); - return ((next >> 16) & 0x7fff).to_string(); + return Cow::Owned(((next >> 16) & 0x7fff).to_string()); } "SRANDOM" => { // $SRANDOM - 32 bits from the OS CSPRNG, not the LCG (bash 5.1). let mut b = [0u8; 4]; if getrandom::fill(&mut b).is_err() { - return String::new(); + return Cow::Owned(String::new()); } - return u32::from_le_bytes(b).to_string(); + return Cow::Owned(u32::from_le_bytes(b).to_string()); } // $LINENO - current line number from command span. A // `local LINENO` is an ordinary variable and `unset LINENO` // makes it ordinary (bash). "LINENO" if !self.lineno_unset && !self.is_local_anywhere("LINENO") => { - return self.current_line.to_string(); + return Cow::Owned(self.current_line.to_string()); } // `PWD=x` / `unset PWD` hold until the directory changes // (bash keeps PWD an ordinary variable that `cd` rewrites). "PWD" => { if self.pwd_shadow.as_ref() == Some(&self.cwd) { - return self - .scoped - .variables - .get("PWD") - .cloned() - .unwrap_or_default(); + return Cow::Owned( + self.scoped + .variables + .get("PWD") + .cloned() + .unwrap_or_default(), + ); } - return self.cwd.to_string_lossy().to_string(); + return Cow::Owned(self.cwd.to_string_lossy().to_string()); } "OLDPWD" => { if let Some(v) = self.scoped.variables.get("OLDPWD") { - return v.clone(); + return Cow::Owned(v.clone()); } - return self.cwd.to_string_lossy().to_string(); + return Cow::Owned(self.cwd.to_string_lossy().to_string()); } "HOSTNAME" => { if let Some(v) = self.scoped.variables.get("HOSTNAME") { - return v.clone(); + return Cow::Owned(v.clone()); } - return "localhost".to_string(); + return Cow::Owned("localhost".to_string()); } "BASH_VERSION" => { - return COMPAT_BASH_VERSION.to_string(); + return Cow::Owned(COMPAT_BASH_VERSION.to_string()); } "BASH_SUBSHELL" => { - return self.bash_subshell.to_string(); + return Cow::Owned(self.bash_subshell.to_string()); } // The enabled `set -o` / `shopt` options, colon-separated. - "SHELLOPTS" => return builtins::shellopts_value(&self.scoped.variables), - "BASHOPTS" => return builtins::bashopts_value(&self.scoped.variables), + "SHELLOPTS" => return Cow::Owned(builtins::shellopts_value(&self.scoped.variables)), + "BASHOPTS" => return Cow::Owned(builtins::bashopts_value(&self.scoped.variables)), "SECONDS" => { let (start, base) = self.seconds_base; let elapsed = i64::try_from(start.elapsed().as_secs()).unwrap_or(i64::MAX); - return base.saturating_add(elapsed).to_string(); + return Cow::Owned(base.saturating_add(elapsed).to_string()); } _ => {} } @@ -14321,21 +14331,23 @@ impl Interpreter { // $0 is the script/shell name; functions and `source` // do not change it. if let Some(frame) = self.call_stack.iter().rev().find(|f| !f.keeps_arg0) { - return frame.name.clone(); + return Cow::Owned(frame.name.clone()); } - return Self::DEFAULT_ARG0.to_string(); + return Cow::Owned(Self::DEFAULT_ARG0.to_string()); } // $1, $2, etc. (1-indexed) if let Some(frame) = self.call_stack.last() && n > 0 && n <= frame.positional.len() { - return frame.positional[n - 1].clone(); + return Cow::Owned(frame.positional[n - 1].clone()); } - return String::new(); + return Cow::Owned(String::new()); } - self.lookup_regular_variable(name).unwrap_or_default() + self.lookup_regular_variable_ref(name) + .map(Cow::Borrowed) + .unwrap_or_default() } /// Check if a variable is set (for `set -u` / nounset). diff --git a/crates/bashkit/src/limits.rs b/crates/bashkit/src/limits.rs index b6e8bc403..09917b4dc 100644 --- a/crates/bashkit/src/limits.rs +++ b/crates/bashkit/src/limits.rs @@ -1200,6 +1200,17 @@ pub(crate) struct BudgetedString { } impl BudgetedString { + /// Adopt existing storage without copying, retaining its capacity charge. + pub(crate) fn try_from_string( + inner: String, + budget: Option<&ExecutionBudget>, + ) -> Result { + let lease = budget + .map(|budget| budget.lease_bytes(inner.capacity())) + .transpose()?; + Ok(Self { inner, lease }) + } + pub(crate) fn new(budget: Option<&ExecutionBudget>) -> Result { Ok(Self { inner: String::new(), diff --git a/crates/bashkit/tests/integration/execution_budget_tests.rs b/crates/bashkit/tests/integration/execution_budget_tests.rs index 66343d14e..dda05244c 100644 --- a/crates/bashkit/tests/integration/execution_budget_tests.rs +++ b/crates/bashkit/tests/integration/execution_budget_tests.rs @@ -523,3 +523,112 @@ async fn cdpath_search_yields_to_cancellation() { ); assert!(result.unwrap_err().to_string().contains("cancelled")); } + +#[tokio::test] +async fn indexed_array_pending_values_share_live_byte_limit() { + for assignment in [ + "a=([0]=$v [0]=$v [0]=$v [0]=$v)", + "declare -a a=([0]=$v [0]=$v [0]=$v [0]=$v)", + "a+=([0]=$v [0]=$v [0]=$v [0]=$v)", + "a=(\"$v\" \"$v\" \"$v\" \"$v\")", + "a=(\"$v\"{1..4})", + "a=([0]=$v$v$v)", + ] { + let mut bash = Bash::builder() + .limits(ExecutionLimits::new().max_live_intermediate_bytes(8_192)) + .build(); + bash.exec(&format!("v='{}'; a=(original)", "x".repeat(4_096))) + .await + .unwrap(); + assert_budget_exhausted(bash.exec(assignment).await); + assert_eq!( + bash.exec("echo ${a[0]}").await.unwrap().stdout, + "original\n" + ); + assert_eq!(bash.exec("echo reused").await.unwrap().stdout, "reused\n"); + } +} + +#[tokio::test] +async fn indexed_array_pending_keys_share_live_byte_limit() { + let mut bash = Bash::builder() + .limits(ExecutionLimits::new().max_live_intermediate_bytes(8_192)) + .build(); + // Leading zeroes keep every subscript valid and small. + bash.exec(&format!("k='{}'", "0".repeat(4_096))) + .await + .unwrap(); + assert_budget_exhausted(bash.exec("a=([$k]=x [$k]=y [$k]=z)").await); +} + +#[tokio::test] +async fn indexed_array_pending_storage_is_released_between_assignments() { + let mut bash = Bash::builder() + .limits(ExecutionLimits::new().max_live_intermediate_bytes(8_192)) + .build(); + let result = bash + .exec(&format!( + "v='{}'; a=([0]=$v); a=([0]=$v); a=([0]=$v); echo ${{#a[0]}}", + "x".repeat(4_096) + )) + .await + .unwrap(); + assert_eq!(result.stdout, "4096\n"); +} + +#[tokio::test] +async fn indexed_array_budget_preserves_expansion_order() { + let script = "a=(old); a=([0]=1+2 [a[0]]=$a [0]+=x); printf '%s\\n' \"${a[0]}\" \"${a[3]}\"; a+=([0]=new [1]=$a); printf '%s\\n' \"${a[1]}\""; + let expected = std::process::Command::new("bash") + .args(["--noprofile", "--norc", "-c", script]) + .output() + .unwrap(); + assert!(expected.status.success()); + let mut bash = Bash::builder() + .limits(ExecutionLimits::new().max_live_intermediate_bytes(8_192)) + .build(); + let result = bash.exec(script).await.unwrap(); + assert_eq!(result.exit_code, 0); + assert_eq!(result.stdout.as_bytes(), expected.stdout); + assert_eq!(result.stderr.as_bytes(), expected.stderr); +} + +#[tokio::test] +async fn indexed_array_empty_pending_items_still_charge_container_storage() { + let mut bash = Bash::builder() + .limits(ExecutionLimits::new().max_live_intermediate_bytes(256)) + .build(); + let script = format!("a=({})", "[0]='' ".repeat(64)); + assert_budget_exhausted(bash.exec(&script).await); +} + +#[tokio::test] +async fn indexed_array_duplicate_subscripts_remain_valid_at_entry_limit() { + let mut bash = Bash::builder() + .memory_limits(bashkit::MemoryLimits::new().max_array_entries(1)) + .limits(ExecutionLimits::new().max_live_intermediate_bytes(8_192)) + .build(); + let result = bash + .exec(&format!( + "v='{}'; a=([0]=$v [0]=$v [0]=$v); echo ${{#a[@]}} ${{#a[0]}}", + "x".repeat(1_024) + )) + .await + .unwrap(); + assert_eq!(result.stdout, "1 1024\n"); +} + +#[tokio::test] +async fn indexed_array_substitutions_release_consumed_output() { + let mut bash = Bash::builder() + .limits(ExecutionLimits::new().max_live_intermediate_bytes(7_168)) + .build(); + bash.exec(&format!("v='{}'", "x".repeat(1_024))) + .await + .unwrap(); + let result = bash + .exec("a=([0]=$(printf %s \"$v\")$(printf %s \"$v\")$(printf %s \"$v\")); echo ${#a[0]}") + .await + .unwrap(); + assert_eq!(result.stdout, "3072\n"); +} diff --git a/docs/security.md b/docs/security.md index 91dc142b8..29a035fd8 100644 --- a/docs/security.md +++ b/docs/security.md @@ -29,6 +29,12 @@ normalization storage is reserved against `max_live_intermediate_bytes` before allocation. Searches also consume the shared work budget and check cancellation and deadlines, including when every candidate is missing. +Indexed compound-array assignments expand values before evaluating subscripts. +Pending keys, values, fields and container capacity count against +`max_live_intermediate_bytes`, including repeated writes to the same index. +Exceeding that budget returns a resource-limit error even when the final array +would fit the retained-variable budget. + Command hash entries (`hash -p FILE NAME...` and automatic PATH lookups) charge each retained name and pathname against the same live-byte budget before copying. Entries that survive between executions count toward the next execution's budget. diff --git a/knowledge/security/threat-model.md b/knowledge/security/threat-model.md index db87172e1..546366f80 100644 --- a/knowledge/security/threat-model.md +++ b/knowledge/security/threat-model.md @@ -280,7 +280,7 @@ runaway scripts without permanently breaking the session. | TM-DOS-063 | Persistent fd exhaustion | `exec N>/tmp/f` across many `N` values, or `cmd {v}>/tmp/f` in a loop (bash keeps `{var}` fds open), grows `exec_fd_table`/coproc fd buffers for the session | `ExecutionLimits::max_file_descriptors` (default 1024) caps persistent custom descriptors; reused fds and standard 0/1/2 don't count. A command's own `N<= 3) fd takes a slot for that command only and a refused open closes the ones it already opened (`here_doc_fds_count_against_fd_cap_and_close`). A `coproc` needs two free slots and also takes a background-job slot (TM-DOS-122); its pipes hold at most 4 KiB plus one command's output before the writer waits | **MITIGATED** | | TM-DOS-109 | Silent truncation at builtin caps | A builtin that stops at a resource cap but exits 0 with no diagnostic (awk loops/recursion/getline caps, `seq`, `yes`, `numfmt`, `history`), or an awk `sprintf` expression that reports a format cap yet returns an empty value, gives scripts and agents incomplete output as success (#2446, #2456, like sed #1005) | Reaching a builtin cap is never silent: `builtins::limits::cap_exceeded` keeps the partial output, writes `: limit () exceeded; output truncated` and exits 1; awk caps and formatting errors are gawk-style fatal errors (`awk: fatal: ...`, exit 2, END skipped). Regression tests: `builtin_cap_security_tests` | **MITIGATED** | | TM-DOS-110 | In-builtin memory growth aborts the host | A value that keeps growing inside awk (`s = s s`, `gsub` with a long replacement, `sprintf("%s%s%s", s, s, s)`, `$1000000000 = 1`, filling arrays, deep recursion shadowing large values) or jq (`until(false; . + .)`, `"x" * 1e12`, `[range(1e12)]`, a non-emitting `until(false; .)` growing evaluator state) allocates until the allocator fails and the embedding process aborts with SIGABRT (#2444) | awk: every string is checked against `AWK_MAX_STRING_BYTES` (16 MiB) *before* it is allocated (concat, `SUBSEP` keys, `sub`/`gsub`/`gensub` as the result grows, `sprintf`/`printf`, `print` joins, rebuilt `$0`); `$N` assignment is capped at `AWK_MAX_FIELD_INDEX`; variables, array elements and values parked in call frames are accounted and capped at `ExecutionLimits::max_live_intermediate_bytes` (default 32 MB); each is a fatal awk error (exit 2) and the script carries on. jq: jaq-json is vendored with a live-memory meter charging every string and array/object body to the same limit, checked before growth (`jq: error: value size limit (N bytes) exceeded`, exit 5), sticky across `try`; value operations poll the execution deadline so non-emitting loops stop at the timeout; output conversion is capped by `max_stdout_bytes`; `--stream` events are produced incrementally with shared ancestor keys, metered cursor/path/event bodies and pre-allocation checks, including `--slurp`, and every traversal step checks shared work/cancellation/deadline limits. Regression tests: stream memory/work coverage in `execution_budget_tests` and `memory_growth_security_tests`. Jq evaluator contexts are now counted (64 live-context ceiling) and every context clone polls the deadline, so pure tail recursion times out and non-tail recursion fails before host stack exhaustion (#2465). The unwind-based abort requires `panic = "unwind"`; panic-abort embedders need process isolation for untrusted jq | **MITIGATED** | -| TM-DOS-114 | Arrays bypass the retained-memory byte budget | `max_array_entries` counted entries but never their keys or values, so `x` doubled to 4 MiB and parked in ten array slots retained ~40 MB past the 10 MB variable-byte budget and exited 0 (#2462) | Array keys and values charge `max_total_variable_bytes` alongside scalars (`MemoryBudget::array_bytes`; indexed keys are machine integers already covered by the entry count, associative keys are attacker-controlled so they count). Element writes, `+=` appends, bulk `a=(...)` replacement, and `declare -A` compound writes are checked before insertion; bytes are released on element unset, whole-array unset, replacement, and local scope pop. An over-budget write records the first memory-limit error and fails execution (TM-DOS-111 mechanism) rather than dropping the value; reaching `max_array_entries` keeps its established silent-skip behaviour (TM-DOS-060). Regression tests: `array_budget_security_tests` | **MITIGATED** | +| TM-DOS-114 | Arrays bypass the retained-memory byte budget | `max_array_entries` counted entries but never their keys or values, so `x` doubled to 4 MiB and parked in ten array slots retained ~40 MB past the 10 MB variable-byte budget and exited 0 (#2462) | Array keys and values charge `max_total_variable_bytes` alongside scalars (`MemoryBudget::array_bytes`; indexed keys are machine integers already covered by the entry count, associative keys are attacker-controlled so they count). Element writes, `+=` appends, bulk `a=(...)` replacement, and `declare -A` compound writes are checked before insertion; bytes are released on element unset, whole-array unset, replacement, and local scope pop. An over-budget write records the first memory-limit error and fails execution (TM-DOS-111 mechanism) rather than dropping the value; reaching `max_array_entries` keeps its established silent-skip behaviour (TM-DOS-060). Indexed compound assignments also keep expanded keys, values, fields and their container capacity leased against `max_live_intermediate_bytes` until consumed; duplicate indices cannot evade this transient cap. Scalar storage is borrowed during expansion so pending copies reserve capacity before allocation. Consumed command-substitution output releases its temporary lease once the pending word buffer owns the bytes. Value-first expansion and ordered subscript evaluation remain intact. Regression tests: `array_budget_security_tests`, `execution_budget_tests::indexed_array_*` | **MITIGATED** | | TM-DOS-115 | `$(