diff --git a/CHANGELOG.md b/CHANGELOG.md index 573db46..e79daf5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,102 @@ # Changelog +## Unreleased + +### Added + +- `map_size` of a `HashMap`, `MapSet.size`, and `length` are Rust's `len()`, + a `usize` where one is expected; so is arithmetic on them there. +- `Enum.empty?/1` and comparing with `[]` or `%{}` are `is_empty()`. +- `%{}` and `%{key => value}` build a `HashMap` or `BTreeMap` where one is + expected. +- A generator over a map or set behind a reference iterates it by reference, + copying copy keys and values and borrowing the rest. +- `first..last//step` with a positive literal step counts up between dynamic + bounds, as Rust's `first..=last` does. +- `Enum.sort/1` sorts booleans, strings, and tuples as well as integers. +- A generator over a map or set is typed, map and set specs keep their key and + value types, and an Elixir integer bound with `=` takes the integer type its + later uses need. +- `%{}` where a reference to a map is expected builds one and borrows it, and a + map key of unknown type is borrowed. +- A closure that only calls a function with its argument is that function. +- An interpolated string is `format!` of its parts, and a `for ... reduce:` + over several generators or filters reduces the comprehension of the + variables its clauses use. +- `MapSet.new()` where a reference to a set is expected is borrowed, a + fallible list concatenated propagates its error, and cloning a reference to + a reference clones the value. +- `List.replace_at/3` computes the new item, then sets it in place, leaving the + list as it is for an index past its end. +- `left ++ right` concatenates lists, growing `left` in place at its last use: + `list ++ [item]` pushes, and `list ++ other` extends. +- A comprehension's type follows all its generators, `{:some, value}` and + `{:ok, value}` patterns included, and a tuple bound with `=` types each + variable. +- A `Vec` read from Rust source is a list, as one a spec names is. +- A `case`, `if`, `cond`, or block has the type of its first branch whose type + is known, and a block's bindings are typed in order. +- `Enum.map(list, fn item -> item.clone() end)` is `cloned()` over a list of + references, and the list itself otherwise. +- `Enum.reduce/3` builds its accumulator as the expected type, `%{}` and `[]` + included, and has the reducer's type when its initial value has none. +- Indexing a map or list, and the module's own aliases named inside other + types, are typed; a `Vec`, reference, or tuple rebuilt from Rust syntax keeps + its kind; an alias of a copy type is copied. +- Copy items a slice lends are copied out for any use, a tuple pattern on a + reference binds references to its elements, Elixir's integer compares in + the Rust integer type of the other side, and a function argument that + cannot fail to match is bound with `=` rather than matched. +- A variable put into a new list or pushed while still in use is copied, and + `Map.get/3` with an empty default is `unwrap_or_default()`. +- `String.downcase/1` and `String.upcase/1` lower to `to_lowercase()` and + `to_uppercase()`, and membership among literals, as `word in ["no", "off"]`, + asks an array, looking a `String` up as a `&str`. +- A comprehension over a list of tuples still in use iterates it by + reference, copying copy elements rather than cloning every item. +- `MapSet.new/1` of listed items is `HashSet::from([...])`, and a list literal + where a slice is expected is a borrowed array, as `&[]`. +- A `for ... reduce:` whose one clause binds the accumulator to a variable + names the accumulator after it, typed, rather than matching it each turn. +- Comparing an option with `nil` is `is_some()` or `is_none()`. + +### Fixed + +- What indexing reaches is never taken from its collection; it is copied. +- Taking a value behind a reference copies it, so an update to a borrowed set + never changes the set it was given. +- A list matched against a list pattern binds its items by reference, and a + slice index read from one is copied. +- `Map.update/4` matches the present value, so the function and the default + may each take what they use. +- A map is indexed by a borrowed key. +- Collecting into a slice argument collects a `Vec` and borrows it. +- `Enum.concat/2` and `Enum.zip/2` take a list only at its last use, and copy + from one the other side borrows. +- An iterator collected into a list and iterated again is iterated as it is. +- A generator over one value computes it before the item moves into the pair. +- A string literal key is looked up as the `&str` it is, and comparing a string + with `""` asks whether it is empty. +- Operators read borrowed copy values a pattern binds: a comparison derefs the + borrowed side, `and`/`or` and `if` take booleans, and an atom compared with + an enum is that enum's variant. +- A count passed where a `u32` or another integer is expected is cast to it. +- `List.first/1` and `List.last/1` on an iterator collected into a list take + from the iterator. +- A nested generator over a list an earlier item holds iterates it by + reference instead of moving it out of a borrow, and a generator over a slice + types its items as references. +- A `for ... reduce:` types the items it binds, so a borrowed copy item is read + where it is stored. +- An option field reached through a reference is matched by reference, binding + a reference to what it holds. +- A binding keeps the option or result type its right-hand side gives rather + than what a later use suggests, and a tuple destructured from a fallible call + types its elements. +- An empty map whose type is left to Rust is `HashMap::new()`. +- A range bound that can fail propagates its error, and a range a method is + called on is parenthesized, as `(1..=count).into_iter()`. + ## 1.0.0-rc.19 - 2026-10-26 ### Added diff --git a/lib/rustq/meta/ast.ex b/lib/rustq/meta/ast.ex index b2f39f4..10d406e 100644 --- a/lib/rustq/meta/ast.ex +++ b/lib/rustq/meta/ast.ex @@ -769,6 +769,7 @@ defmodule RustQ.Meta.AST do :=, :!, :.., + :..//, :!=, :!==, :%, diff --git a/lib/rustq/meta/core/call.ex b/lib/rustq/meta/core/call.ex index f539692..b33917f 100644 --- a/lib/rustq/meta/core/call.ex +++ b/lib/rustq/meta/core/call.ex @@ -22,6 +22,7 @@ defmodule RustQ.Meta.Core.Call do :rem, :in, :.., + :..//, :abs, :min, :max, diff --git a/lib/rustq/meta/desugar.ex b/lib/rustq/meta/desugar.ex index 30f0996..c38d09e 100644 --- a/lib/rustq/meta/desugar.ex +++ b/lib/rustq/meta/desugar.ex @@ -7,7 +7,11 @@ defmodule RustQ.Meta.Desugar do comprehension collecting into it; * `Enum.group_by/2,3` is a map filled by a loop, and `Enum.uniq/1` and `Enum.uniq_by/2` keep the items a set has not seen; - * a generator over `Enum.with_index(list)` enumerates the list in place. + * a generator over `Enum.with_index(list)` enumerates the list in place; + * `left ++ right` is `Enum.concat(left, right)`; + * an interpolated string is `format!` of its parts; + * a `for ... reduce:` over several generators or filters reduces the comprehension of the + variables they bind. Where a rewrite needs what only lowering knows, it leaves an internal form for lowering to resolve: `__rustq_items__(list)` iterates a list as its ownership allows, and @@ -15,6 +19,8 @@ defmodule RustQ.Meta.Desugar do cloned otherwise. """ + alias RustQ.Meta.Pattern + @doc "Rewrites `body`'s standard-library calls that are other Rusty-Elixir." @spec desugar(Macro.t()) :: Macro.t() def desugar(body), do: Macro.prewalk(body, &rewrite/1) @@ -32,8 +38,19 @@ defmodule RustQ.Meta.Desugar do end) end - def apply_function({:fn, _, [{:->, _, [[pattern], body]}]}, value), - do: quote(do: case(unquote(value), do: (unquote(pattern) -> unquote(body)))) + # A pattern that cannot fail, as a tuple of variables, binds the value; another matches it. + def apply_function({:fn, _, [{:->, _, [[pattern], body]}]}, value) do + if binding?(pattern), + do: + quote( + do: + ( + unquote(pattern) = unquote(value) + unquote(body) + ) + ), + else: quote(do: case(unquote(value), do: (unquote(pattern) -> unquote(body)))) + end def apply_function({:&, _, [{:/, _, [{{:., _, _} = remote, _, _}, 1]}]}, value), do: {remote, [], [value]} @@ -50,12 +67,22 @@ defmodule RustQ.Meta.Desugar do ) end + defp binding?({name, _, ast_context}) when is_atom(name) and is_atom(ast_context), do: true + defp binding?({left, right}), do: binding?(left) and binding?(right) + defp binding?({:{}, _, elements}), do: Enum.all?(elements, &binding?/1) + defp binding?(_pattern), do: false + # Names a rewrite introduces, which shadow nothing a body writes. defp fresh(name), do: Macro.var(:"rustq_#{name}", __MODULE__) defp rewrite({{:., _, [{:__aliases__, _, [:Map]}, :new]}, _, [enumerable | transform]}), do: comprehension(enumerable, transform, quote(do: HashMap.new())) + # A set of listed items is built from them as they are; see `RustQ.Meta.Lower.MapSet`. + defp rewrite({{:., _, [{:__aliases__, _, [:MapSet]}, :new]}, _, [items]} = call) + when is_list(items), + do: call + defp rewrite({{:., _, [{:__aliases__, _, [:MapSet]}, :new]}, _, [enumerable | transform]}), do: comprehension(enumerable, transform, quote(do: HashSet.new())) @@ -113,9 +140,70 @@ defmodule RustQ.Meta.Desugar do end end - defp rewrite({:for, meta, args}), do: {:for, meta, Enum.map(args, &indexed_generator/1)} + defp rewrite({:++, _, [left, right]}), do: quote(do: Enum.concat(unquote(left), unquote(right))) + + defp rewrite({:<<>>, _, parts} = binary) do + if Enum.all?(parts, &(is_binary(&1) or interpolated(&1) != nil)) do + format = Enum.map_join(parts, &format_part/1) + {:format!, [], [format | for(part <- parts, not is_binary(part), do: interpolated(part))]} + else + binary + end + end + + defp rewrite({:for, meta, args}) do + {qualifiers, options} = Enum.split_while(args, &(not Keyword.keyword?(&1) or &1 == [])) + options = Enum.concat(options) + + if Keyword.has_key?(options, :reduce) and match?([_, _ | _], qualifiers), + do: reduced(qualifiers, options), + else: {:for, meta, Enum.map(args, &indexed_generator/1)} + end + defp rewrite(ast), do: ast + defp reduced(qualifiers, options) do + clauses = Keyword.fetch!(options, :do) + + item = + case bound_variables(qualifiers, clauses) do + [variable] -> variable + variables -> {:{}, [], variables} + end + + items = {:for, [], qualifiers ++ [[do: item]]} + + quote( + do: + Enum.reduce( + unquote(items), + unquote(Keyword.fetch!(options, :reduce)), + unquote(reducer(item, clauses)) + ) + ) + end + + # The variables the generators bind that the clauses use, in order. + defp bound_variables(qualifiers, clauses) do + used = clauses |> Pattern.variables() |> MapSet.new(&variable_key/1) + + qualifiers + |> Enum.flat_map(&generator_variables/1) + |> Enum.uniq_by(&variable_key/1) + |> Enum.filter(&MapSet.member?(used, variable_key(&1))) + end + + defp generator_variables({:<-, _, [pattern, _collection]}), do: Pattern.variables(pattern) + defp generator_variables(_filter), do: [] + + defp variable_key({name, _meta, context}), do: {name, context} + + defp reducer(item, [{:->, _, [[accumulator], body]}]), + do: quote(do: fn unquote(item), unquote(accumulator) -> unquote(body) end) + + defp reducer(item, clauses), + do: quote(do: fn unquote(item), rustq_acc -> case(rustq_acc, do: unquote(clauses)) end) + # `{item, index} <- Enum.with_index(list)` enumerates the list, binding `{index, item}`, # rather than collecting the pairs first. defp indexed_generator( @@ -154,6 +242,16 @@ defmodule RustQ.Meta.Desugar do end end + defp interpolated({:"::", _, [{{:., _, [Kernel, :to_string]}, _, [value]}, {:binary, _, _}]}), + do: value + + defp interpolated(_part), do: nil + + defp format_part(text) when is_binary(text), + do: text |> String.replace("{", "{{") |> String.replace("}", "}}") + + defp format_part(_interpolation), do: "{}" + # The collection an empty collectable names, or nil for one with items. defp empty_collection({:%{}, _, []}), do: quote(do: HashMap.new()) defp empty_collection([]), do: quote(do: Vec.new()) diff --git a/lib/rustq/meta/lower.ex b/lib/rustq/meta/lower.ex index 10d55a5..5d4f55b 100644 --- a/lib/rustq/meta/lower.ex +++ b/lib/rustq/meta/lower.ex @@ -700,6 +700,55 @@ defmodule RustQ.Meta.Lower do when is_list(fields), do: lower_struct_update(base, fields, context) + # A map literal where a reference to a map is expected builds the map and borrows it. + defp lower_expected_expr_context( + {:%{}, _, entries} = map, + %Type{kind: :ref, meta: %{inner: %Type{} = inner}}, + %Context{} = context + ) + when is_list(entries) do + case Collection.keyed(inner) do + {:map, _key, _value} -> A.ref(lower_expected_expr_context(map, inner, context)) + _other -> lower_expr(map, context) + end + end + + # A map literal where a map's alias is expected builds the map it names. + defp lower_expected_expr_context( + {:%{}, _, entries} = map, + %Type{kind: :alias, meta: %{target: %Type{ast: %AST.TypePath{parts: parts}} = target}}, + %Context{} = context + ) + when is_list(entries) and parts in [[:HashMap], [:BTreeMap]], + do: lower_expected_expr_context(map, target, context) + + # A map literal where a map is expected builds one: `%{}` is `HashMap::new()`, and + # `%{key => value}` is `HashMap::from([(key, value)])`. + defp lower_expected_expr_context( + {:%{}, _, entries}, + %Type{ast: %AST.TypePath{parts: parts, generics: [_key, _value]}} = expected_type, + %Context{} = context + ) + when is_list(entries) and parts in [[:HashMap], [:BTreeMap]] do + case entries do + [] -> + A.path_call(parts ++ [:new]) + + entries -> + {:map, key, value} = Collection.keyed(expected_type) + + pairs = + Enum.map(entries, fn {entry_key, entry_value} -> + A.tuple([ + lower_expr(entry_key, key, context), + lower_expr(entry_value, value, context) + ]) + end) + + A.path_call(parts ++ [:from], [A.array(pairs)]) + end + end + defp lower_expected_expr_context( {:%{}, _, fields}, %Type{kind: :struct, ast: %AST.TypePath{parts: parts}} = expected_type, @@ -801,11 +850,14 @@ defmodule RustQ.Meta.Lower do if slice_type?(expected_type), do: A.ref(array), else: array end + # A list literal where a slice is expected is a borrowed array, which allocates nothing. defp lower_expected_expr_context(values, %Type{} = expected_type, %Context{} = context) when is_list(values) do - vec = A.vec(lower_expected_collection_values(values, expected_type, context)) + values = lower_expected_collection_values(values, expected_type, context) - if slice_type?(expected_type), do: A.ref(vec), else: vec + if slice_type?(expected_type), + do: A.ref(%AST.ArrayLiteral{values: values}), + else: A.vec(values) end defp lower_expected_expr_context( @@ -850,6 +902,34 @@ defmodule RustQ.Meta.Lower do ), do: lower_pipe(left, right, expected_type, context) + # Arithmetic where an integer is expected computes in that integer, as its operands do. + defp lower_expected_expr_context( + {operator, _, [left, right]}, + %Type{kind: kind} = expected_type, + %Context{} = context + ) + when operator in [:+, :-, :*] and + kind in [:i8, :i16, :i32, :i64, :isize, :u8, :u16, :u32, :u64, :usize] do + A.binary( + lower_expected_operand(left, expected_type, context), + operator_op(operator), + lower_expected_operand(right, expected_type, context) + ) + end + + # A count is a `usize` where one is expected, not Elixir's integer. + defp lower_expected_expr_context( + {function, _, [_value] = args} = call, + %Type{} = expected_type, + context + ) + when function in [:length, :byte_size, :map_size] do + case BindingIndex.get(context.callables, nil, function, length(args)) do + %RustQ.Binding.Callable{} -> lower_local_or_macro_call(function, args, context) + nil -> Stdlib.lower!(call, stdlib_context(context, expected_type)) + end + end + defp lower_expected_expr_context( {{:., _, [receiver = {:__aliases__, _, [module]}, function]}, _, args} = call, %Type{} = expected_type, @@ -941,11 +1021,18 @@ defmodule RustQ.Meta.Lower do defp lower_expected_collection_values(values, expected_type, context) do case collection_item_type(expected_type) do - %Type{} = item_type -> Enum.map(values, &lower_expr(&1, item_type, context)) + %Type{} = item_type -> Enum.map(values, &lower_list_item(&1, item_type, context)) nil -> Enum.map(values, &lower_array_value(&1, context)) end end + # A value put in a new list while still in use, or still in its collection, is copied in. + defp lower_list_item(value, %Type{} = item_type, context) do + if in_use?(value, context) and not Collection.copy?(item_type), + do: A.method(lower_expr(value, item_type, context), :clone), + else: lower_expr(value, item_type, context) + end + @doc "The item type of a list a comprehension builds." def collection_item_type(type) do Type.vec_inner(Type.expected_value(type)) || Type.slice_inner(type) @@ -1075,7 +1162,7 @@ defmodule RustQ.Meta.Lower do do: %{lower_closure_args(args, body, context) | move: true} defp lower_expr_context({:fn, _, [{:->, _, [args, body]}]}, %Context{} = context), - do: lower_closure_args(args, body, context) + do: args |> lower_closure_args(body, context) |> eta_reduced() defp lower_expr_context( {{:., _, [receiver = {:__aliases__, _, [module]}, function]}, _, args} = call, @@ -1117,6 +1204,9 @@ defmodule RustQ.Meta.Lower do when is_list(fields), do: lower_struct_update(base, fields, context) + # An empty map whose type is left to Rust is a `HashMap` that later uses type. + defp lower_expr_context({:%{}, _, []}, %Context{}), do: A.path_call([:HashMap, :new]) + defp lower_expr_context( {:__rustq_typed__, _, [expression, %Type{} = type]}, %Context{} = context @@ -1160,13 +1250,16 @@ defmodule RustQ.Meta.Lower do do: A.index(lower_expr(receiver, context), lower_expr(index, context)) # `container[key]` indexes; a map is indexed by a reference to its key. + # A map is indexed by a borrowed key. defp lower_expr_context({{:., _, [Access, :get]}, _, [receiver, key]}, %Context{} = context) do - index = lower_expr(key, context) + receiver_type = + infer_expr_type(receiver, context.vars) || Typing.synth(receiver, typing_env(context)) index = - if map_type?(infer_expr_type(receiver, context.vars)), - do: A.ref(index), - else: index + case Collection.keyed(receiver_type) do + {:map, _key, _value} -> borrowed_key(key, context) + _list -> lower_expr(key, RustQ.Spec.type(quote(do: RustQ.Type.usize())), context) + end A.index(lower_expr(receiver, context), index) end @@ -1182,6 +1275,7 @@ defmodule RustQ.Meta.Lower do defp lower_expr_context({function, _, args} = call, %Context{} = context) when function in [ :.., + :..//, :in, :==, :!=, @@ -1518,12 +1612,19 @@ defmodule RustQ.Meta.Lower do end end + # An option field reached through a reference is matched by reference, as what it holds + # cannot be moved out. defp case_scrutinee(expression, %Type{} = expression_type, clauses, %Context{} = context) do - if propagate_case_scrutinee?(expression_type, clauses) do - inner = Type.inner(expression_type) - {inner, lower_checked_expr(expression, inner, context)} - else - {expression_type, lower_expr(expression, context)} + cond do + propagate_case_scrutinee?(expression_type, clauses) -> + inner = Type.inner(expression_type) + {inner, lower_checked_expr(expression, inner, context)} + + lent_field?(expression, expression_type, context) -> + {Type.ref(expression_type), A.ref(lower_expr(expression, context))} + + true -> + {expression_type, lower_expr(expression, context)} end end @@ -1601,8 +1702,14 @@ defmodule RustQ.Meta.Lower do end end - defp lower_condition(condition, %Context{} = context), - do: lower_binary_operand(condition, context) + # A condition is a boolean; a borrowed one is read. + defp lower_condition(condition, %Context{} = context) do + case infer_expr_type(condition, context.vars) || + Typing.synth(condition, typing_env(context)) do + %Type{kind: :ref} -> A.deref(lower_expr(condition, context)) + _owned -> lower_binary_operand(condition, context) + end + end defp lower_with(clauses, %Context{} = context, expected_type \\ nil) do {matches, body_opts} = Enum.split_while(clauses, &match?({:<-, _, _}, &1)) @@ -1864,6 +1971,19 @@ defmodule RustQ.Meta.Lower do end end + # An option field holding what cannot be copied, reached through a reference. + defp lent_field?(expression, %Type{kind: :option} = type, %Context{} = context) do + case root_variable_node(expression) do + {name, _meta, _context} -> + behind_reference?(expression, name, context) and not Collection.copy?(Type.inner(type)) + + nil -> + false + end + end + + defp lent_field?(_expression, _type, _context), do: false + defp behind_reference?({name, _, ast_context}, name, _context) when is_atom(ast_context), do: false @@ -1956,10 +2076,9 @@ defmodule RustQ.Meta.Lower do receiver_type = infer_expr_type(receiver, context.vars) || Typing.synth(receiver, typing_env(context)) - %AST.MethodCall{ - receiver: lower_dot_receiver(receiver, :into_iter, context), - method: borrowed_iteration(:into_iter, [], receiver_type) - } + receiver + |> lower_dot_receiver(:into_iter, context) + |> Collection.iterated(borrowed_iteration(:into_iter, [], receiver_type)) true -> %AST.MethodCall{ @@ -1978,6 +2097,13 @@ defmodule RustQ.Meta.Lower do ), do: A.try(lower_expr(receiver, context)) + defp lower_dot_receiver(receiver, :clone, %Context{} = context) do + receiver_type = + infer_expr_type(receiver, context.vars) || Typing.synth(receiver, typing_env(context)) + + lower_method_receiver(receiver, receiver_type, :clone, context) + end + defp lower_dot_receiver(receiver, _method, %Context{} = context), do: lower_expr(receiver, context) @@ -2038,6 +2164,14 @@ defmodule RustQ.Meta.Lower do # A fallible receiver of a method its result lacks, as `decoder.read_bool().encode(env)`, # propagates its error first, as a fallible operand does. + # Cloning a reference to a reference clones what it refers to, the value Elixir means. + defp lower_method_receiver(receiver, %Type{kind: :ref} = receiver_type, :clone, context) do + case Type.ref_inner(receiver_type) do + %Type{kind: :ref} -> A.deref(lower_expr(receiver, context)) + _value -> lower_expr(receiver, context) + end + end + defp lower_method_receiver(receiver, receiver_type, function, %Context{} = context) do if fallible_receiver?(receiver_type, function, context), do: A.try(lower_expr(receiver, context)), @@ -2338,6 +2472,40 @@ defmodule RustQ.Meta.Lower do ) end + # A closure that only calls a function with its argument is that function, as clippy prefers: + # `|item| Wrapped(item)` is `Wrapped`. + defp eta_reduced(%AST.Closure{ + args: [name], + body: %AST.PathCall{path: path, args: [%AST.Var{name: name}], generics: []}, + move: false + }) + when is_atom(name), + do: path + + defp eta_reduced(%AST.Closure{ + args: [name], + body: %AST.LocalCall{name: function, args: [%AST.Var{name: name}]}, + move: false + }) + when is_atom(name), + do: A.path([function]) + + defp eta_reduced(closure), do: closure + + # A map's key is read by reference; one already a reference is passed as it is. + defp borrowed_key(key, %Context{} = context) do + case infer_expr_type(key, context.vars) || Typing.synth(key, typing_env(context)) do + %Type{kind: kind} when kind in [:ref, :mut_ref, :slice] -> lower_expr(key, context) + _owned -> A.ref(lower_expr(key, context)) + end + end + + defp lower_expected_operand(expression, %Type{} = expected_type, %Context{} = context) do + if fallible_expression?(expression, context) and propagation_allowed?(context.return_type), + do: A.try(lower_expr(expression, context)), + else: lower_expr(expression, expected_type, context) + end + defp lower_binary_op(left, op, right, %Context{} = context) do A.binary(lower_binary_operand(left, context), op, lower_binary_operand(right, context)) end @@ -2349,6 +2517,18 @@ defmodule RustQ.Meta.Lower do # A collection is in use unless this is its value's last use; see `RustQ.Meta.Ownership`. A # field reached through a reference is never taken, since the reference does not own it. def in_use?(collection, %Context{} = context) do + indexed_place?(collection) or root_in_use?(collection, context) + end + + # What indexing reaches stays in its collection, so it is never taken. + defp indexed_place?({{:., _, [Access, :get]}, _, [_collection, _key]}), do: true + + defp indexed_place?({{:., _, [receiver, _field]}, meta, []}), + do: meta[:no_parens] == true and indexed_place?(receiver) + + defp indexed_place?(_expression), do: false + + defp root_in_use?(collection, %Context{} = context) do case root_variable_node(collection) do {name, _meta, _context} = variable -> behind_reference?(collection, name, context) or @@ -2472,14 +2652,6 @@ defmodule RustQ.Meta.Lower do [A.return_stmt(lower_expr(final, %{context | position: :return}))] end - defp map_type?(%Type{kind: kind} = type) when kind in [:ref, :mut_ref], - do: map_type?(Type.ref_inner(type)) - - defp map_type?(%Type{rust: rust}) when is_binary(rust), - do: String.starts_with?(rust, ["HashMap<", "BTreeMap<"]) - - defp map_type?(_type), do: false - @doc "A captured named function, as the Rust path of that function." def lower_function_capture({name, _, capture_context}, %Context{}) when is_atom(name) and is_atom(capture_context), diff --git a/lib/rustq/meta/lower/collection.ex b/lib/rustq/meta/lower/collection.ex index 6b90d83..76b5d82 100644 --- a/lib/rustq/meta/lower/collection.ex +++ b/lib/rustq/meta/lower/collection.ex @@ -28,6 +28,17 @@ defmodule RustQ.Meta.Lower.Collection do def keyed(%Type{kind: :alias, meta: %{target: %Type{} = target}}), do: keyed(target) + def keyed(%Type{ + ast: %AST.TypePath{parts: parts}, + meta: %{generics: [%Type{} = key, %Type{} = value]} + }) + when parts in [[:HashMap], [:BTreeMap]], + do: {:map, key, value} + + def keyed(%Type{ast: %AST.TypePath{parts: parts}, meta: %{generics: [%Type{} = item]}}) + when parts in [[:HashSet], [:BTreeSet]], + do: {:set, item} + def keyed(%Type{ast: %AST.TypePath{parts: parts, generics: [key, value]}}) do if List.last(parts) in @maps, do: {:map, Type.ast_type(key), Type.ast_type(value)} end @@ -80,16 +91,55 @@ defmodule RustQ.Meta.Lower.Collection do @spec owned(Macro.t(), Context.t()) :: AST.expr() def owned(collection, %Context{} = context) do lowered = lowered(collection, context) + type = type_of(collection, context) - if in_use?(collection, context), - do: A.method(lowered, :clone), - else: lowered + cond do + borrowed?(type) -> A.method(lowered, :clone) + in_use?(collection, context) and not copy?(type) -> A.method(lowered, :clone) + true -> lowered + end end - @doc "A key or item passed by reference, borrowed unless it already is one." + # A value behind a reference is never owned: taking it copies what it refers to. + defp borrowed?(%Type{kind: kind}) when kind in [:ref, :mut_ref, :slice], do: true + defp borrowed?(_type), do: false + + @doc """ + A key or item passed by reference, borrowed unless it already is one, as a string literal + is. + """ @spec borrowed(Macro.t(), Type.t(), Context.t()) :: AST.expr() - def borrowed(expression, %Type{} = type, %Context{} = context), - do: lowered(expression, Type.ref(type), context) + def borrowed(expression, %Type{}, %Context{} = context) when is_binary(expression), + do: lowered(expression, context) + + def borrowed(expression, %Type{} = type, %Context{} = context) do + case type_of(expression, context) do + nil -> A.ref(lowered(expression, context)) + _known -> lowered(expression, Type.ref(type), context) + end + end + + @doc """ + `expression` iterated with `method`; what an iterator collects into a list is iterated as + the iterator it is. + """ + @spec iterated(AST.expr(), atom()) :: AST.expr() + def iterated(expression, method) do + if collected?(expression), do: expression.receiver, else: A.method(expression, method) + end + + @doc """ + Whether an expression collects an iterator into a list, or into whatever is needed, not a map + or set, which keeps one item per key. + """ + @spec collected?(AST.expr()) :: boolean() + def collected?(%AST.MethodCall{method: :collect, generics: []}), do: true + def collected?(%AST.MethodCall{method: :collect, generics: [%AST.TypeVec{}]}), do: true + + def collected?(%AST.MethodCall{method: :collect, generics: [%AST.TypePath{parts: [:Vec]}]}), + do: true + + def collected?(_expression), do: false @doc "A value read out of a collection: copied when that costs nothing, otherwise cloned." @spec read_out(AST.expr(), Type.t()) :: AST.expr() @@ -104,6 +154,7 @@ defmodule RustQ.Meta.Lower.Collection do """ @spec copy?(Type.t() | nil) :: boolean() def copy?(%Type{kind: kind}) when kind in [:ref], do: true + def copy?(%Type{kind: :alias, meta: %{target: %Type{} = target}}), do: copy?(target) def copy?(%Type{} = type) do case Type.category(type) do diff --git a/lib/rustq/meta/lower/comprehension.ex b/lib/rustq/meta/lower/comprehension.ex index ee419fe..8e8fc6c 100644 --- a/lib/rustq/meta/lower/comprehension.ex +++ b/lib/rustq/meta/lower/comprehension.ex @@ -10,6 +10,7 @@ defmodule RustQ.Meta.Lower.Comprehension do alias RustQ.Diagnostic alias RustQ.Meta.Lower + alias RustQ.Meta.Lower.Collection alias RustQ.Meta.Lower.Env, as: Context alias RustQ.Meta.Lower.Pattern, as: LowerPattern alias RustQ.Meta.Pattern @@ -120,16 +121,11 @@ defmodule RustQ.Meta.Lower.Comprehension do {nested?, generated} ) do {filters, rest} = Enum.split_while(rest, &(not generator?(&1))) - item_type = comprehension_item_type(collection, context) - root = root_variable(collection) - reused? = - root != nil and - ((nested? and root not in generated) or - (item_type != nil and root not in generated and - Lower.in_use?(collection, context))) + {collection, item_type} = generator_items(collection, context) + {iteration, item_type} = iteration(collection, item_type, {nested?, generated}, context) - {stage, binder} = generator_stage(pattern, collection, filters, item_type, reused?) + {stage, binder} = generator_stage(pattern, collection, filters, item_type, iteration) bound = LowerPattern.context_with_match_pattern(pattern, item_type, context) generated = LowerPattern.binding_pattern_names(pattern) ++ generated @@ -142,18 +138,19 @@ defmodule RustQ.Meta.Lower.Comprehension do # A generator over one value, as `{:some, id} <- [record_id(record)]`, matches that value # beside the item instead of iterating a list made for it. + # The value comes first, so it may borrow what the item moves into the pair. [{:<-, meta, [value_pattern, [value]]} | later] -> paired = quote( do: unquote(stage) |> map(fn unquote(typed_arg(binder, binder_type)) -> - {unquote(binder), unquote(value)} + {unquote(value), unquote(binder)} end) ) comprehension_pipeline( - [{:<-, meta, [{binder, value_pattern}, paired]} | later], + [{:<-, meta, [{value_pattern, binder}, paired]} | later], body, bound, {nested?, generated} @@ -189,17 +186,97 @@ defmodule RustQ.Meta.Lower.Comprehension do defp untyped(body), do: body # A generator over a list gives its closures the list's items, which they may then use as - # that type; other collections leave their items untyped. + # that type, and one over a slice references to them; other collections leave their items + # untyped. defp comprehension_item_type(collection, %Context{} = context) do unless iterator_call?(collection) do case Lower.infer_expr_type(collection, context.vars) || Typing.synth(collection, Lower.typing_env(context)) do %Type{kind: :vec} = type -> Type.vec_inner(type) + %Type{kind: :slice} = type -> Type.ref(Type.slice_inner(type)) _unknown -> nil end end end + # What a generator iterates, and the type of each item. + defp generator_items(collection, %Context{} = context), + do: + borrowed_keyed(collection, context) || borrowed_tuples(collection, context) || + {collection, comprehension_item_type(collection, context)} + + # A map or set behind a reference, or still in use, is iterated by reference: its keys and + # values copied where that costs nothing, and borrowed otherwise, so a record is never + # cloned to be read. + defp borrowed_keyed(collection, %Context{} = context) do + with false <- iterator_call?(collection), + %Type{} = type <- + Lower.infer_expr_type(collection, context.vars) || + Typing.synth(collection, Lower.typing_env(context)), + keyed when keyed != nil <- Collection.keyed(type), + true <- Type.ref_inner(type) != nil or Lower.in_use?(collection, context) do + borrowed_items(collection, keyed) + else + _owned -> nil + end + end + + # A list of tuples still in use is iterated by reference as a map is, so reading a copy + # element never clones the others. + defp borrowed_tuples(collection, %Context{} = context) do + with false <- iterator_call?(collection), + %Type{kind: :vec} = type <- + Lower.infer_expr_type(collection, context.vars) || + Typing.synth(collection, Lower.typing_env(context)), + %Type{} = item <- Type.vec_inner(type), + {:tuple, elements} <- Type.category(item), + false <- Collection.copy?(item), + true <- Lower.in_use?(collection, context) do + vars = Enum.map(1..length(elements)//1, &Macro.var(:"element#{&1}", __MODULE__)) + read = Enum.zip_with(vars, elements, &read_item/2) + + items = + quote do + unquote(collection) + |> iter() + |> map(fn {unquote_splicing(vars)} -> {unquote_splicing(read)} end) + end + + {items, Type.tuple(Enum.map(elements, &item_type/1))} + else + _owned -> nil + end + end + + defp borrowed_items(collection, {:map, key, value}) do + {key_var, value_var} = {Macro.var(:key, __MODULE__), Macro.var(:value, __MODULE__)} + + items = + quote do + unquote(collection) + |> iter() + |> map(fn {unquote(key_var), unquote(value_var)} -> + {unquote(read_item(key_var, key)), unquote(read_item(value_var, value))} + end) + end + + {items, Type.tuple([item_type(key), item_type(value)])} + end + + defp borrowed_items(collection, {:set, item}) do + items = + if Collection.copy?(item), + do: quote(do: unquote(collection) |> iter() |> copied()), + else: quote(do: unquote(collection) |> iter()) + + {items, item_type(item)} + end + + defp read_item(var, type), + do: if(Collection.copy?(type), do: quote(do: deref(unquote(var))), else: var) + + defp item_type(type), do: if(Collection.copy?(type), do: type, else: Type.ref(type)) + # The type of the variables a generator binds, as one variable or a tuple of them. defp binder_type({name, _, ast_context}, %Context{vars: vars}) when is_atom(ast_context), do: Map.get(vars, name) @@ -278,11 +355,43 @@ defmodule RustQ.Meta.Lower.Comprehension do # A later generator over a collection from outside the comprehension iterates it once per # item of the earlier ones, so it copies its items rather than consuming it, as does a # generator over a collection still in use or reached through a reference. - defp generator_stage(pattern, collection, filters, item_type, reused?) do + # How a generator iterates its collection, and the type of each item. + defp iteration(collection, item_type, {nested?, generated}, %Context{} = context) do + root = root_variable(collection) + + cond do + lent_field?(collection, root, item_type, generated, context) -> + {:lent, Type.ref(item_type)} + + reused?(collection, root, item_type, {nested?, generated}, context) -> + {:reused, item_type} + + true -> + {:owned, item_type} + end + end + + # A list field of an item an earlier generator borrows is lent: iterated by reference, as a + # slice is. + defp lent_field?(collection, root, item_type, generated, context) do + item_type != nil and match?({{:., _, [_receiver, _field]}, _, []}, collection) and + root in generated and lent?(root, context) + end + + # A collection an enclosing generator would iterate again, or still in use, is copied from. + defp reused?(_collection, nil, _item_type, _scope, _context), do: false + + defp reused?(collection, root, item_type, {nested?, generated}, context) do + root not in generated and + (nested? or (item_type != nil and Lower.in_use?(collection, context))) + end + + defp generator_stage(pattern, collection, filters, item_type, iteration) do items = cond do iterator_call?(collection) -> collection - reused? -> quote(do: unquote(collection) |> iter() |> cloned()) + iteration == :lent -> quote(do: unquote(collection) |> iter()) + iteration == :reused -> quote(do: unquote(collection) |> iter() |> cloned()) true -> quote(do: unquote(collection) |> into_iter()) end @@ -304,6 +413,9 @@ defmodule RustQ.Meta.Lower.Comprehension do defp root_variable(_collection), do: nil + defp lent?(name, %Context{vars: vars}), + do: match?(%Type{kind: kind} when kind in [:ref, :mut_ref], Map.get(vars, name)) + # A collection already given as an iterator, as in `values.iter()`, is iterated as it is. defp iterator_call?({{:., _, [_receiver, method]}, _, args}) when is_list(args), do: method in @iterator_methods @@ -383,15 +495,29 @@ defmodule RustQ.Meta.Lower.Comprehension do %Type{} = return_type, %Context{} = context ) do - acc = A.var(:__rustq_reduce) acc_type = reduce_acc_type(initial, return_type) + context = with_items(pattern, expression, context) + + {acc, update} = + case clauses do + # A single clause binding the accumulator to a variable names the accumulator. + [{:->, _, [[{name, _, ast_context} = variable], body]}] + when is_atom(name) and is_atom(ast_context) -> + {A.var(name), reduce_body(variable, body, acc_type, context)} + + clauses -> + acc = A.var(:__rustq_reduce) + {acc, A.match_expr(acc, lower_for_reduce_arms(clauses, acc_type, context))} + end A.block_expr([ - A.let_mut(acc.name, lower_reduce_initial(initial, acc_type, context)), + A.let_mut(acc.name, lower_reduce_initial(initial, acc_type, context), + type: annotated(acc_type) + ), A.for_( LowerPattern.lower_binding_pattern(pattern), Lower.lower_expr(expression, context), - [A.assign(acc, A.match_expr(acc, lower_for_reduce_arms(clauses, acc_type, context)))] + [A.assign(acc, update)] ), A.return_stmt(acc) ]) @@ -406,6 +532,11 @@ defmodule RustQ.Meta.Lower.Comprehension do ) end + # The accumulator is annotated with its type, which an empty initial value leaves to Rust, + # unless it is the result a fallible body propagates. + defp annotated(%Type{} = acc_type), + do: if(Type.propagates?(acc_type), do: nil, else: acc_type.ast) + defp lower_reduce_initial(initial, %Type{} = acc_type, %Context{} = context) do if Type.propagates?(acc_type) do Lower.lower_return_expr(initial, acc_type, context) @@ -427,27 +558,48 @@ defmodule RustQ.Meta.Lower.Comprehension do defp reduce_acc_type(_initial, %Type{} = return_type), do: return_type + # The loop binds the generator's items, typed as the collection lends them. + defp with_items(pattern, collection, %Context{} = context) do + case comprehension_item_type(collection, context) do + %Type{} = item_type -> LowerPattern.context_with_match_pattern(pattern, item_type, context) + nil -> context + end + end + + # An accumulator no clause matches is carried on, unless a clause matches any. defp lower_for_reduce_arms(clauses, %Type{} = return_type, %Context{} = context) do carry = :__rustq_reduce_value - Enum.map(clauses, fn {:->, _, [[pattern], body]} -> - body_context = LowerPattern.context_with_match_pattern(pattern, return_type, context) + carried = + if Enum.any?(clauses, fn {:->, _, [[pattern], _body]} -> catch_all?(pattern) end), + do: [], + else: [%AST.Arm{pattern: %AST.PatVar{name: carry}, body: [A.return_stmt(A.var(carry))]}] + Enum.map(clauses, fn {:->, _, [[pattern], body]} -> %AST.Arm{ pattern: LowerPattern.lower_match_pattern(pattern, return_type, context), - body: - Lower.lower_clause_body( - body, - %{body_context | position: :return, return_type: return_type}, - return_type - ) + body: lower_reduce_clause(pattern, body, return_type, context) } - end) ++ - [ - %AST.Arm{ - pattern: %AST.PatVar{name: carry}, - body: [A.return_stmt(A.var(carry))] - } - ] + end) ++ carried end + + defp reduce_body(pattern, body, %Type{} = return_type, %Context{} = context) do + case lower_reduce_clause(pattern, body, return_type, context) do + [%AST.Return{expr: expr}] -> expr + statements -> A.block_expr(statements) + end + end + + defp lower_reduce_clause(pattern, body, %Type{} = return_type, %Context{} = context) do + body_context = LowerPattern.context_with_match_pattern(pattern, return_type, context) + + Lower.lower_clause_body( + body, + %{body_context | position: :return, return_type: return_type}, + return_type + ) + end + + defp catch_all?({name, _, ast_context}) when is_atom(name) and is_atom(ast_context), do: true + defp catch_all?(_pattern), do: false end diff --git a/lib/rustq/meta/lower/enum.ex b/lib/rustq/meta/lower/enum.ex index 90061a0..693ec5b 100644 --- a/lib/rustq/meta/lower/enum.ex +++ b/lib/rustq/meta/lower/enum.ex @@ -56,13 +56,16 @@ defmodule RustQ.Meta.Lower.Enum do |> integer_count(context)} end - defp lower_call(%Call{function: :empty?, args: [collection]}, %Context{} = context), - do: - {:ok, - collection - |> iterate(:question, context) - |> A.method(:next) - |> A.method(:is_none)} + # A collection is asked directly; a stage, whether it yields anything. + defp lower_call(%Call{function: :empty?, args: [collection]}, %Context{} = context) do + case stage(collection) do + {:ok, stage} -> + {:ok, stage |> continued(context) |> A.method(:next) |> A.method(:is_none)} + + :error -> + {:ok, collection |> lower_collection(context) |> A.method(:is_empty)} + end + end defp lower_call(%Call{function: :member?, args: [collection, value]}, %Context{} = context), do: @@ -95,23 +98,28 @@ defmodule RustQ.Meta.Lower.Enum do |> A.method(:flatten) |> A.method(:collect)} - defp lower_call(%Call{function: :concat, args: [left, right]}, %Context{} = context), - do: + # A list taken whole grows in place: `list ++ [item]` pushes the item, and `list ++ other` + # extends the list with the other's items. + defp lower_call(%Call{function: :concat, args: [left, right]}, %Context{} = context) do + if owned_list?(left, context) do + grown = Collection.updated(left, &appended(&1, right, left, context), context) + {:ok, if(match?(%Type{kind: :slice}, context.expected), do: A.ref(grown), else: grown)} + else {:ok, left - |> lowered(context) - |> A.method(:into_iter) - |> A.method(:chain, [lowered(right, context)]) - |> A.method(:collect)} + |> iterate(:keeps_items, context) + |> A.method(:chain, [iterable(right, left, context)]) + |> collect(context)} + end + end defp lower_call(%Call{function: :zip, args: [left, right]}, %Context{} = context), do: {:ok, left - |> lowered(context) - |> A.method(:into_iter) - |> A.method(:zip, [lowered(right, context)]) - |> A.method(:collect)} + |> iterate(:keeps_items, context) + |> A.method(:zip, [iterable(right, left, context)]) + |> collect(context)} defp lower_call(%Call{function: :unzip, args: [collection]}, %Context{} = context), do: @@ -129,27 +137,17 @@ defmodule RustQ.Meta.Lower.Enum do |> A.method(:rev) |> A.method(:collect)} + # Rust orders integers, booleans, strings, and tuples of them as Elixir does. defp lower_call(%Call{function: :sort, args: [collection]} = call, %Context{} = context) do - case collection_inner(type_of(collection, context)) do - %Type{} = inner - when inner.kind in [:i8, :i16, :i32, :i64, :isize, :u8, :u16, :u32, :u64, :usize] -> - name = :rustq_sorted_values - variable = A.var(name) - - {:ok, - A.block_expr([ - %AST.Let{pattern: %AST.PatVar{name: name}, expr: lowered(collection, context)}, - A.stmt(A.method(variable, :sort)), - A.return_stmt(variable) - ])} - - _type -> - Diagnostic.lower( - :unsupported_enum_sort_semantics, - call.source, - "Enum.sort/1 currently requires a statically typed integer collection", - suggestion: "Use an explicit comparator adapter for other ordering semantics." - ) + if ordered?(collection_inner(type_of(collection, context))) do + {:ok, Collection.updated(collection, &[A.stmt(A.method(&1, :sort))], context)} + else + Diagnostic.lower( + :unsupported_enum_sort_semantics, + call.source, + "Enum.sort/1 requires a list of integers, booleans, strings, or tuples of them", + suggestion: "Sort by a key with Enum.sort_by/2 for other ordering semantics." + ) end end @@ -431,11 +429,12 @@ defmodule RustQ.Meta.Lower.Enum do def synth(%Call{function: :sum, args: [collection]}, %TypeContext{} = context), do: collection |> type_of(context) |> collection_inner_result() + # A reduction has its initial value's type, or, for an empty literal, the reducer's. def synth( - %Call{function: :reduce, args: [_collection, initial, _reducer]}, + %Call{function: :reduce, args: [_collection, initial, reducer]}, %TypeContext{} = context ), - do: type_result(type_of(initial, context)) + do: type_result(type_of(initial, context) || reducer_type(reducer, context)) def synth(%Call{function: operation}, %TypeContext{}) when operation in [:any?, :all?], do: {:ok, Spec.type(quote(do: boolean()))} @@ -596,6 +595,26 @@ defmodule RustQ.Meta.Lower.Enum do def synth(%Call{}, %TypeContext{}), do: :unsupported + # Cloning each item copies the list's items: what references point to is cloned with + # `cloned()`, as clippy prefers to a map, and owned items are already their own copies. + defp lower_map( + collection, + {:fn, _, + [ + {:->, _, + [[{name, _, ast_context}], {{:., _, [{name, _, ast_context}, :clone]}, _, []}]} + ]}, + context + ) + when is_atom(name) and is_atom(ast_context) do + iterator = iterate(collection, :keeps_items, context) + + case collection_inner(type_of(collection, context)) do + %Type{kind: :ref} -> iterator |> A.method(:cloned) |> collect(context) + _owned -> collect(iterator, context) + end + end + defp lower_map(collection, {:fn, _, [{:->, _, [args, body]}]}, context) do mapped_type = collection_inner(context.expected) iteration = iteration(collection, :maps_items, context) @@ -638,11 +657,23 @@ defmodule RustQ.Meta.Lower.Enum do ) do iteration = iteration(collection, :maps_items, context) [item] = typed_items([item], collection, iteration, context) - closure = closure([accumulator, item], body, context) + + # The accumulator is what the reduction returns, where that is known. + {initial, closure} = + case context.expected do + %Type{} = acc_type -> + accumulator = {:__rustq_typed_arg__, [], [accumulator, acc_type]} + + {lowered(initial, acc_type, context), + closure([accumulator, item], body, acc_type, context)} + + nil -> + {lowered(initial, context), closure([accumulator, item], body, context)} + end collection |> iterate_as(iteration, context) - |> A.method(:fold, [lowered(initial, context), closure]) + |> A.method(:fold, [initial, closure]) end defp lower_reduce(_collection, _initial, other, _context) do @@ -774,6 +805,11 @@ defmodule RustQ.Meta.Lower.Enum do defp mapper_type(_mapper, _inner, _context), do: nil + defp reducer_type({:fn, _, [{:->, _, [[_item, _accumulator], body]}]}, context), + do: type_of(body, context) + + defp reducer_type(_reducer, _context), do: nil + defp collection_inner(%Type{} = type), do: Type.vec_inner(type) || Type.slice_inner(type) defp collection_inner(_type), do: nil @@ -878,9 +914,11 @@ defmodule RustQ.Meta.Lower.Enum do case iteration do :owned -> - if borrowed_collection?(type_of(collection, context)), - do: A.method(receiver, :iter), - else: A.method(receiver, :into_iter) + cond do + borrowed_collection?(type_of(collection, context)) -> A.method(receiver, :iter) + Collection.collected?(receiver) -> receiver.receiver + true -> A.method(receiver, :into_iter) + end :borrowed -> A.method(receiver, :iter) @@ -908,6 +946,15 @@ defmodule RustQ.Meta.Lower.Enum do do: list_iteration(collection, type, use, context) defp iteration(%Type{} = type, _collection, :keeps_items, _context), do: lent_iteration(type) + + # Copy items lent by a slice are copied out for any use, as `iter().copied()`. + defp iteration(%Type{} = type, _collection, _use, _context) do + case Type.slice_inner(type) || collection_inner(type) do + %Type{} = inner -> if Collection.copy?(inner), do: :copied, else: :owned + nil -> :owned + end + end + defp iteration(_type, _collection, _use, _context), do: :owned defp list_iteration(collection, type, use, context) do @@ -946,8 +993,61 @@ defmodule RustQ.Meta.Lower.Enum do %AST.MethodCall{receiver: receiver, method: :collect, args: [], generics: [type]} end + # Where a slice is expected, the items collect into a list it borrows. + defp collect(receiver, %Context{expected: %Type{kind: :slice} = expected}) do + vec = expected |> Type.slice_inner() |> Type.vec() + A.ref(%AST.MethodCall{receiver: receiver, method: :collect, args: [], generics: [vec.ast]}) + end + defp collect(receiver, %Context{}), do: A.method(receiver, :collect) + # What `chain` and `zip` take: any iterable, so a list taken whole is passed as it is. The + # list the iterator before it borrows is still borrowed, so it is copied from too; a list a + # fallible call gives propagates its error first. + defp iterable(collection, before, context) do + iterator = + cond do + same_root?(collection, before) -> iterate_as(collection, :cloned, context) + fallible?(collection, context) -> collection |> operand(context) |> A.method(:into_iter) + true -> iterate(collection, :keeps_items, context) + end + + case iterator do + %AST.MethodCall{method: :into_iter, receiver: list, args: []} -> list + iterator -> iterator + end + end + + defp owned_list?(list, context) do + match?(%Type{kind: :vec}, type_of(list, context)) and + iteration(list, :keeps_items, context) == :owned + end + + defp appended(list, items, _before, context) when is_list(items), + do: Enum.map(items, &A.stmt(A.method(list, :push, [Collection.owned(&1, context)]))) + + defp appended(list, other, before, context), + do: [A.stmt(A.method(list, :extend, [iterable(other, before, context)]))] + + defp ordered?(%Type{kind: :alias, meta: %{target: %Type{} = target}}), do: ordered?(target) + + defp ordered?(%Type{} = type) do + case Type.category(type) do + category when category in [:integer, :boolean, :string] -> true + {:tuple, elements} -> Enum.all?(elements, &ordered?/1) + _other -> false + end + end + + defp ordered?(nil), do: false + + defp fallible?(collection, context), + do: + match?(%Type{kind: kind} when kind in [:result, :nif_result], type_of(collection, context)) + + defp same_root?({name, _, context}, {name, _, context}) when is_atom(context), do: true + defp same_root?(_collection, _before), do: false + defp continuation(body) do Macro.prewalk(body, fn {:cont, value} -> quote(do: ok(unquote(value))) diff --git a/lib/rustq/meta/lower/kernel.ex b/lib/rustq/meta/lower/kernel.ex index 60c3271..d124840 100644 --- a/lib/rustq/meta/lower/kernel.ex +++ b/lib/rustq/meta/lower/kernel.ex @@ -7,6 +7,7 @@ defmodule RustQ.Meta.Lower.Kernel do use RustQ.Meta.Lower.Stdlib.Lowering alias RustQ.Diagnostic + alias RustQ.Meta.Lower.Collection alias RustQ.Meta.Type alias RustQ.Spec @@ -30,8 +31,9 @@ defmodule RustQ.Meta.Lower.Kernel do @comparison_functions [:==, :!=, :<, :<=, :>, :>=, :and, :or, :not, :is_nil] @impl true - def lower(%Call{function: function} = call, %Context{} = context) when function in [:.., :in], - do: RustQ.Meta.Lower.Range.lower(call, context) + def lower(%Call{function: function} = call, %Context{} = context) + when function in [:.., :..//, :in], + do: RustQ.Meta.Lower.Range.lower(call, context) def lower(%Call{function: :/, args: [left, right]} = call, %Context{} = context) do if integer_type?(type_of(left, context)) or integer_type?(type_of(right, context)) do @@ -44,18 +46,41 @@ defmodule RustQ.Meta.Lower.Kernel do end end + # Comparing a collection or string with an empty one asks whether it is empty. + def lower(%Call{function: function, args: [collection, empty]}, %Context{} = context) + when function in [:==, :!=] and empty in [[], ""], + do: {:ok, emptiness(function, collection, context)} + + def lower(%Call{function: function, args: [collection, {:%{}, _, []}]}, %Context{} = context) + when function in [:==, :!=], + do: {:ok, emptiness(function, collection, context)} + + # Comparing an option with `nil` asks whether it holds a value, which needs no equality on + # what it holds. + def lower(%Call{function: function, args: [option, nil]}, %Context{} = context) + when function in [:==, :!=] do + method = if function == :==, do: :is_none, else: :is_some + {:ok, A.method(lowered(option, context), method)} + end + # Comparing two options or results compares the wrappers; neither side propagates. def lower(%Call{function: function, args: [left, right]}, %Context{} = context) when function in [:==, :!=] do - lower_operand = - if wrapper_comparison?(left, right, context), - do: &lowered(&1, context), - else: &operand(&1, context) - - {:ok, - A.binary(lower_operand.(left), Map.fetch!(@binary_ops, function), lower_operand.(right))} + if wrapper_comparison?(left, right, context), + do: + {:ok, + A.binary( + lowered(left, context), + Map.fetch!(@binary_ops, function), + lowered(right, context) + )}, + else: {:ok, compared(function, left, right, context)} end + def lower(%Call{function: function, args: [left, right]}, %Context{} = context) + when function in @comparison_functions and is_map_key(@binary_ops, function), + do: {:ok, compared(function, left, right, context)} + def lower(%Call{function: function, args: [left, right]}, %Context{} = context) when is_map_key(@binary_ops, function) do {:ok, @@ -91,11 +116,16 @@ defmodule RustQ.Meta.Lower.Kernel do do: {:ok, value |> lowered(context) |> A.method(:len) |> integer_count(context)} def lower(%Call{function: :map_size, args: [map]} = call, %Context{} = context) do - case struct_fields(shape_type(type_of(map, context))) do - fields when is_list(fields) -> - evaluated_constant(map, length(fields), context) + type = shape_type(type_of(map, context)) - _unknown -> + cond do + match?({:map, _key, _value}, Collection.keyed(type)) -> + {:ok, map |> lowered(context) |> A.method(:len) |> integer_count(context)} + + is_list(struct_fields(type)) -> + evaluated_constant(map, length(struct_fields(type)), context) + + true -> unsupported_semantics!(call, "map_size/1 requires a statically typed map or struct") end end @@ -150,7 +180,7 @@ defmodule RustQ.Meta.Lower.Kernel do @impl true def synth(%Call{function: function} = call, %TypeContext{} = context) - when function in [:.., :in], + when function in [:.., :..//, :in], do: RustQ.Meta.Lower.Range.synth(call, context) def synth(%Call{function: function}, %TypeContext{}) when function in @comparison_functions, @@ -204,6 +234,45 @@ defmodule RustQ.Meta.Lower.Kernel do ) end + # Elixir's integer, as a count is, compares in the Rust integer type of the other side. + # A boolean operator takes booleans, and a comparison values of one type: a borrowed copy + # operand is read where the other is not borrowed or the operator is boolean, and an atom + # or integer literal takes the other operand's type. + defp compared(function, left, right, context) do + {left_type, right_type} = {type_of(left, context), type_of(right, context)} + logical? = function in [:and, :or] + + A.binary( + compared_operand(left, left_type, right_type, logical?, context), + Map.fetch!(@binary_ops, function), + compared_operand(right, right_type, left_type, logical?, context) + ) + end + + defp compared_operand(operand, %Type{kind: :i64}, %Type{kind: kind} = other, _logical?, context) + when kind in [:u8, :u16, :u32, :u64, :usize, :i8, :i16, :i32, :isize], + do: lowered(operand, other, context) + + defp compared_operand(operand, _type, %Type{} = other, false, context) + when is_atom(operand) and operand not in [nil, true, false], + do: lowered(operand, other, context) + + defp compared_operand(operand, %Type{kind: :ref} = type, other, logical?, context) do + if Collection.copy?(Type.ref_inner(type)) and (logical? or not borrowed?(other)), + do: A.deref(lowered(operand, context)), + else: operand(operand, context) + end + + defp compared_operand(operand, _type, _other, _logical?, context), do: operand(operand, context) + + defp borrowed?(%Type{kind: kind}), do: kind in [:ref, :mut_ref] + defp borrowed?(_type), do: false + + defp emptiness(function, collection, context) do + is_empty = collection |> lowered(context) |> A.method(:is_empty) + if function == :==, do: is_empty, else: A.not_(is_empty) + end + defp wrapper_comparison?(left, right, %Context{} = context) do wrapper_literal?(left) or wrapper_literal?(right) or (wrapper_kind(type_of(left, context)) != nil and diff --git a/lib/rustq/meta/lower/list.ex b/lib/rustq/meta/lower/list.ex index 6d2b6c0..3aecf4d 100644 --- a/lib/rustq/meta/lower/list.ex +++ b/lib/rustq/meta/lower/list.ex @@ -1,11 +1,12 @@ defmodule RustQ.Meta.Lower.List do @moduledoc """ - Lowers `List.duplicate/2`, `List.flatten/1`, and `List.wrap/1` for `RustQ.Meta.Lower.Stdlib`, - and types such calls. + Lowers `List.duplicate/2`, `List.flatten/1`, `List.wrap/1`, and `List.replace_at/3` for + `RustQ.Meta.Lower.Stdlib`, and types such calls. """ use RustQ.Meta.Lower.Stdlib.Lowering + alias RustQ.Meta.Lower.Collection alias RustQ.Meta.Type @impl true @@ -16,7 +17,7 @@ defmodule RustQ.Meta.Lower.List do {:ok, list |> lowered(context) - |> A.method(:into_iter) + |> Collection.iterated(:into_iter) |> A.method(terminal)} end @@ -27,11 +28,40 @@ defmodule RustQ.Meta.Lower.List do {:ok, list |> lowered(context) - |> A.method(:into_iter) + |> Collection.iterated(:into_iter) |> A.method(terminal) |> A.method(:unwrap_or, [lowered(default, context)])} end + # The new item is computed before the list is taken, as it may read the list; an index past + # the end leaves the list as it is, as in Elixir. + def lower(%Call{function: :replace_at, args: [list, index, value]}, %Context{} = context) do + with {:ok, index} <- list_index(index, context) do + item = collection_inner(type_of(list, context)) + slot = A.var(:rustq_slot) + + {:ok, + A.block_expr([ + %AST.Let{pattern: %AST.PatVar{name: :rustq_item}, expr: lowered(value, item, context)}, + A.return_stmt( + Collection.updated( + list, + fn list -> + [ + %AST.IfLet{ + pattern: %AST.PatSome{pattern: %AST.PatVar{name: :rustq_slot}}, + expr: A.method(list, :get_mut, [index]), + then: [A.assign(A.deref(slot), A.var(:rustq_item))] + } + ] + end, + context + ) + ) + ])} + end + end + def lower(%Call{function: :wrap, args: [nil]}, %Context{}), do: {:ok, A.vec([])} @@ -112,8 +142,21 @@ defmodule RustQ.Meta.Lower.List do end end + def synth(%Call{function: :replace_at, args: [list, _index, _value]}, %TypeContext{} = context), + do: type_result(type_of(list, context)) + def synth(%Call{}, %TypeContext{}), do: :unsupported + # An index of a type not known here, or left to Rust to infer, is the `usize` Rust infers for + # it; a known signed one is not an index. + defp list_index(index, context) do + case type_of(index, context) do + nil -> {:ok, lowered(index, context)} + %Type{ast: %AST.TypeRaw{source: "_"}} -> {:ok, lowered(index, context)} + _known -> nonnegative_count(index, context) + end + end + defp flatten_iterator(iterator, 1), do: iterator defp flatten_iterator(iterator, depth) do diff --git a/lib/rustq/meta/lower/map.ex b/lib/rustq/meta/lower/map.ex index eb5820e..4979f48 100644 --- a/lib/rustq/meta/lower/map.ex +++ b/lib/rustq/meta/lower/map.ex @@ -46,10 +46,7 @@ defmodule RustQ.Meta.Lower.Map do lookup = A.method(lowered(map, context), :get, [Collection.borrowed(key, key_type, context)]) - {:ok, - lookup - |> Collection.read_out(value_type) - |> A.method(:unwrap_or, [lowered(default, value_type, context)])} + {:ok, lookup |> Collection.read_out(value_type) |> or_default(default, value_type, context)} end # A missing key raises in Elixir; indexing panics, which Rustler raises. @@ -106,7 +103,8 @@ defmodule RustQ.Meta.Lower.Map do )} end - # `update/4` gives a present value to `fun` and stores `default` otherwise, unchanged. + # `update/4` gives a present value to `fun` and stores `default` otherwise, unchanged. Each + # is its own arm, so both may take what they use. defp lower_keyed( %Call{function: :update, args: [map, key, default, fun]}, key_type, @@ -114,25 +112,27 @@ defmodule RustQ.Meta.Lower.Map do context ) do key_name = A.var(:rustq_key) - value_name = A.var(:rustq_value) + {present, updated} = update_arm(fun, value_type, context) + + stored = lowered(default, value_type, context) {:ok, Collection.updated( map, fn map -> - updated = - map - |> A.method(:remove, [A.ref(key_name)]) - |> A.method(:map, [update_function(fun, value_type, context)]) - |> A.method(:unwrap_or, [lowered(default, value_type, context)]) + value = + A.match_expr(A.method(map, :remove, [A.ref(key_name)]), [ + %AST.Arm{pattern: %AST.PatSome{pattern: present}, body: [A.return_stmt(updated)]}, + %AST.Arm{pattern: %AST.PatNone{}, body: [A.return_stmt(stored)]} + ]) [ %AST.Let{ pattern: %AST.PatVar{name: :rustq_key}, expr: lowered(key, key_type, context) }, - %AST.Let{pattern: %AST.PatVar{name: :rustq_value}, expr: updated}, - A.stmt(A.method(map, :insert, [key_name, value_name])) + %AST.Let{pattern: %AST.PatVar{name: :rustq_value}, expr: value}, + A.stmt(A.method(map, :insert, [key_name, A.var(:rustq_value)])) ] end, context @@ -162,18 +162,34 @@ defmodule RustQ.Meta.Lower.Map do defp lower_keyed(%Call{}, _key_type, _value_type, _context), do: :unsupported - defp update_function({:fn, _, [{:->, _, [[argument], body]}]}, value_type, context), - do: closure([{:__rustq_typed_arg__, [], [argument, value_type]}], body, context) + # An empty collection is the value's default, as clippy prefers. + defp or_default(option, [], _value_type, _context), do: A.method(option, :unwrap_or_default) + + defp or_default(option, {:%{}, _, []}, _value_type, _context), + do: A.method(option, :unwrap_or_default) + + defp or_default(option, default, value_type, context), + do: A.method(option, :unwrap_or, [lowered(default, value_type, context)]) - defp update_function({:&, _, [{:/, _, [function, 1]}]}, _value_type, context), - do: capture(function, context) + # The present value's pattern, and `fun`'s result for it: a function's body with its argument + # bound to the value, or a captured function called with it. + defp update_arm({:fn, _, [{:->, _, [[{name, _, ast_context}], body]}]}, value_type, context) + when is_atom(name) and is_atom(ast_context) do + bound = %{context | env: %{context.env | vars: Map.put(context.env.vars, name, value_type)}} + {%AST.PatVar{name: name}, lowered(body, value_type, bound)} + end + + defp update_arm({:&, _, [{:/, _, [function, 1]}]}, _value_type, context) do + %AST.Path{} = path = capture(function, context) + {%AST.PatVar{name: :rustq_present}, %AST.PathCall{path: path, args: [A.var(:rustq_present)]}} + end - defp update_function(other, _value_type, _context) do + defp update_arm(other, _value_type, _context) do Diagnostic.lower( :unsupported_map_update_function, other, "unsupported Map.update function in defrust", - suggestion: "Use a one-argument anonymous function or a named one-argument capture." + suggestion: "Use a one-argument anonymous function with a named argument, or a capture." ) end diff --git a/lib/rustq/meta/lower/map_set.ex b/lib/rustq/meta/lower/map_set.ex index b32ac02..a91c772 100644 --- a/lib/rustq/meta/lower/map_set.ex +++ b/lib/rustq/meta/lower/map_set.ex @@ -1,9 +1,10 @@ defmodule RustQ.Meta.Lower.MapSet do @moduledoc """ Lowers `MapSet` functions on a `HashSet` or `BTreeSet` for `RustQ.Meta.Lower.Stdlib`, and - types such calls: `new/0` (`new/1,2` are comprehensions; see `RustQ.Meta.Desugar`), `put`, `delete`, `member?`, `union`, `difference`, `intersection`, - `subset?`, `disjoint?`, `size`, and `to_list`. An update takes the set only at its last use; - see `RustQ.Meta.Lower.Collection`. + types such calls: `new/0`, `new/1` of listed items (other `new/1,2` calls are + comprehensions; see `RustQ.Meta.Desugar`), `put`, `delete`, `member?`, `union`, `difference`, + `intersection`, `subset?`, `disjoint?`, `size`, and `to_list`. An update takes the set only + at its last use; see `RustQ.Meta.Lower.Collection`. """ use RustQ.Meta.Lower.Stdlib.Lowering @@ -12,9 +13,17 @@ defmodule RustQ.Meta.Lower.MapSet do alias RustQ.Meta.Type @impl true + # A new set where a reference to one is expected is borrowed. + def lower(%Call{function: :new, args: []}, %Context{expected: %Type{kind: :ref}}), + do: {:ok, A.ref(A.path_call([:HashSet, :new]))} + def lower(%Call{function: :new, args: []}, %Context{}), do: {:ok, A.path_call([:HashSet, :new])} + # A set of listed items is built from an array of them. + def lower(%Call{function: :new, args: [items]}, %Context{} = context) when is_list(items), + do: {:ok, A.path_call([:HashSet, :from], [A.array(Enum.map(items, &lowered(&1, context)))])} + def lower(%Call{args: [set | _]} = call, %Context{} = context) do case Collection.keyed_of(set, context) do {:set, item} -> lower_set(call, item, context) @@ -85,7 +94,7 @@ defmodule RustQ.Meta.Lower.MapSet do end defp lower_set(%Call{function: :size, args: [set]}, _item, context) do - {:ok, A.cast(A.method(lowered(set, context), :len), A.type(:i64))} + {:ok, set |> lowered(context) |> A.method(:len) |> integer_count(context)} end defp lower_set(%Call{function: :to_list, args: [set]}, item, context) do diff --git a/lib/rustq/meta/lower/pattern.ex b/lib/rustq/meta/lower/pattern.ex index 1f9cc0e..5359118 100644 --- a/lib/rustq/meta/lower/pattern.ex +++ b/lib/rustq/meta/lower/pattern.ex @@ -68,31 +68,13 @@ defmodule RustQ.Meta.Lower.Pattern do @doc "The variables a pattern binds, as one variable or a tuple of them." # The variables a pattern binds, as one variable or a tuple of them. def pattern_binder(pattern) do - case pattern_variables(pattern) do + case ElixirPattern.variables(pattern) do [] -> quote(do: {}) [variable] -> variable variables -> {:{}, [], variables} end end - defp pattern_variables(pattern) do - {_pattern, variables} = - Macro.prewalk(pattern, [], fn - {:^, _meta, _args}, variables -> - {:pinned, variables} - - {name, meta, context} = variable, variables when is_atom(name) and is_atom(context) -> - if name |> Atom.to_string() |> String.starts_with?("_"), - do: {variable, variables}, - else: {variable, [{name, meta, context} | variables]} - - other, variables -> - {other, variables} - end) - - variables |> Enum.reverse() |> Enum.uniq_by(fn {name, _meta, context} -> {name, context} end) - end - @doc "The type a `case` scrutinee has, as its clauses' patterns reveal it." def infer_case_type_from_patterns(clauses) do if Enum.any?(clauses, fn {:->, _, [[pattern], _body]} -> option_pattern?(pattern) end) do @@ -242,9 +224,37 @@ defmodule RustQ.Meta.Lower.Pattern do end end + # A pattern on a reference to an option binds a reference to what it holds, as Rust's + # default binding mode does. + def context_with_match_pattern( + pattern, + %Type{kind: :ref, meta: %{inner: %Type{kind: :option} = option}}, + %Context{} = context + ), + do: context_with_match_pattern(pattern, Type.option(Type.ref(Type.inner(option))), context) + + # A tuple pattern on a reference to a tuple binds references to its elements, as Rust's + # default binding mode does. + def context_with_match_pattern(pattern, %Type{kind: :ref} = type, %Context{} = context) + when is_tuple(pattern) and (tuple_size(pattern) == 2 or elem(pattern, 0) == :{}) do + case Type.ref_inner(type) do + %Type{kind: :tuple, meta: %{elements: types}} = tuple -> + refs = Enum.map(types, &Type.ref/1) + + context_with_match_pattern( + pattern, + %{tuple | meta: %{tuple.meta | elements: refs}}, + context + ) + + _other -> + context_with_match_pattern(pattern, nil, context) + end + end + # A pattern of unknown type still shadows the names it binds; they lose their old types. def context_with_match_pattern(pattern, _type, %Context{} = context) do - names = pattern |> pattern_variables() |> Enum.map(&elem(&1, 0)) + names = pattern |> ElixirPattern.variables() |> Enum.map(&elem(&1, 0)) %{context | vars: Map.drop(context.vars, names)} end @@ -275,11 +285,12 @@ defmodule RustQ.Meta.Lower.Pattern do defp context_with_list_pattern(pattern, %Type{} = type, %Context{} = context) do {elements, rest} = split_list_pattern(pattern) inner = list_item_type(type) - inner = if inner && type.kind == :slice, do: Type.ref(inner), else: inner + # A list is matched as a slice, so its items bind by reference. + item = if inner && type.kind in [:slice, :vec], do: Type.ref(inner), else: inner context = Enum.reduce(elements, context, fn element, acc -> - context_with_match_pattern(element, inner, acc) + context_with_match_pattern(element, item, acc) end) if rest, @@ -421,9 +432,9 @@ defmodule RustQ.Meta.Lower.Pattern do def lower_match_pattern(pattern, %Type{kind: :slice} = type) when is_list(pattern), do: lower_list_pattern(pattern, type) - # A list pattern on a value of unknown type matches it as a slice. - def lower_match_pattern(pattern, nil) - when is_list(pattern) and pattern != [] and not is_tuple(hd(pattern)), + # A list pattern on a value of unknown type matches it as a slice; a keyword list does not. + def lower_match_pattern([head | _tail] = pattern, nil) + when not (is_tuple(head) and tuple_size(head) == 2 and is_atom(elem(head, 0))), do: lower_list_pattern(pattern, nil) def lower_match_pattern([_head | _tail] = pattern, %Type{kind: :vec} = type), diff --git a/lib/rustq/meta/lower/range.ex b/lib/rustq/meta/lower/range.ex index 867f306..b6a6366 100644 --- a/lib/rustq/meta/lower/range.ex +++ b/lib/rustq/meta/lower/range.ex @@ -6,6 +6,7 @@ defmodule RustQ.Meta.Lower.Range do use RustQ.Meta.Lower.Stdlib.Lowering alias RustQ.Diagnostic + alias RustQ.Meta.Type alias RustQ.Spec @impl true @@ -22,21 +23,48 @@ defmodule RustQ.Meta.Lower.Range do :unsupported_stdlib_semantics, call.source, "generated Rust ranges currently require ascending integer literal bounds", - suggestion: "Use an explicit Rust adapter for dynamic or descending ranges." + suggestion: "Count up with an explicit step, as `first..last//1`, for dynamic bounds." ) end end - def lower(%Call{function: :in, args: [value, collection]}, %Context{} = context) do - receiver = - case lowered(collection, context) do - %AST.Range{} = range -> A.block_expr([A.return_stmt(range)]) - expression -> expression - end + # With a positive step, a range counts up and is empty when it ends below its start, as + # Rust's `start..=stop` is, so its bounds need not be literals; a fallible bound propagates + # its error. + def lower(%Call{function: :..//, args: [start, stop, step]} = call, %Context{} = context) do + if is_integer(step) and step > 0 do + range = %AST.Range{ + start: operand(start, context), + stop: operand(stop, context), + inclusive: true + } - {:ok, A.method(receiver, :contains, [A.ref(lowered(value, context))])} + {:ok, if(step == 1, do: range, else: A.method(range, :step_by, [A.lit(step)]))} + else + Diagnostic.lower( + :unsupported_stdlib_semantics, + call.source, + "generated Rust ranges with a step require a positive integer literal step", + suggestion: "Count up with a positive step, as `first..last//1`." + ) + end end + # Membership in a literal list asks an array, which allocates nothing; a string is looked + # up among string literals as the `&str` they are. + def lower(%Call{function: :in, args: [value, items]}, %Context{} = context) + when is_list(items) do + item = + if Enum.all?(items, &is_binary/1) and string?(type_of(value, context)), + do: A.method(lowered(value, context), :as_str), + else: lowered(value, context) + + {:ok, A.method(A.array(Enum.map(items, &lowered(&1, context))), :contains, [A.ref(item)])} + end + + def lower(%Call{function: :in, args: [value, collection]}, %Context{} = context), + do: {:ok, A.method(lowered(collection, context), :contains, [A.ref(lowered(value, context))])} + def lower(%Call{}, %Context{}), do: :unsupported @impl true @@ -50,4 +78,7 @@ defmodule RustQ.Meta.Lower.Range do do: true defp ascending_literal_range?(_start, _stop), do: false + + defp string?(%Type{} = type), do: Type.category(type) == :string + defp string?(_type), do: false end diff --git a/lib/rustq/meta/lower/stdlib/lowering.ex b/lib/rustq/meta/lower/stdlib/lowering.ex index 9e13d1c..d172df1 100644 --- a/lib/rustq/meta/lower/stdlib/lowering.ex +++ b/lib/rustq/meta/lower/stdlib/lowering.ex @@ -148,9 +148,17 @@ defmodule RustQ.Meta.Lower.Stdlib.Lowering do ])} end - @doc "A Rust count as Elixir's integer, an `i64`, unless the call is expected to produce a `usize`." + @doc """ + A Rust count as Elixir's integer, an `i64`, unless the call is expected to produce a `usize` + or another integer type. + """ @spec integer_count(AST.expr(), Context.t()) :: AST.expr() def integer_count(count, %Context{expected: %Type{kind: :usize}}), do: count + + def integer_count(count, %Context{expected: %Type{kind: kind}}) + when kind in [:u8, :u16, :u32, :u64, :i8, :i16, :i32, :isize], + do: A.cast(count, A.type(kind)) + def integer_count(count, %Context{}), do: A.cast(count, A.type(:i64)) @doc "`count` as a `usize`, when it is a literal or unsigned count that cannot be negative." diff --git a/lib/rustq/meta/lower/string.ex b/lib/rustq/meta/lower/string.ex index 6725fb0..855ce78 100644 --- a/lib/rustq/meta/lower/string.ex +++ b/lib/rustq/meta/lower/string.ex @@ -1,7 +1,8 @@ defmodule RustQ.Meta.Lower.String do @moduledoc """ Lowers `String` functions whose Rust counterparts keep their semantics, such as - `String.replace/3` and `String.trim/1` for `RustQ.Meta.Lower.Stdlib`, and types such calls. + `String.replace/3`, `String.trim/1`, and `String.downcase/1` for `RustQ.Meta.Lower.Stdlib`, + and types such calls. """ use RustQ.Meta.Lower.Stdlib.Lowering @@ -9,6 +10,8 @@ defmodule RustQ.Meta.Lower.String do alias RustQ.Meta.Type alias RustQ.Spec + @case_methods %{downcase: :to_lowercase, upcase: :to_uppercase} + @predicate_methods %{ starts_with?: :starts_with, ends_with?: :ends_with, @@ -32,6 +35,18 @@ defmodule RustQ.Meta.Lower.String do |> A.method(:trim) |> A.method(:to_string)} + # Changing the case of a string made only to be changed reads the string it was made from. + def lower(%Call{function: function, args: [string]}, %Context{} = context) + when is_map_key(@case_methods, function) do + receiver = + case lowered(string, context) do + %AST.MethodCall{method: :to_string, receiver: borrowed} -> borrowed + owned -> owned + end + + {:ok, A.method(receiver, Map.fetch!(@case_methods, function))} + end + def lower( %Call{function: :replace, args: [string, pattern, replacement]}, %Context{} = context @@ -77,7 +92,7 @@ defmodule RustQ.Meta.Lower.String do do: {:ok, Spec.type(quote(do: boolean()))} def synth(%Call{function: function}, %TypeContext{}) - when function in [:trim, :replace, :duplicate], + when function in [:trim, :replace, :duplicate, :downcase, :upcase], do: {:ok, Spec.type(quote(do: String.t()))} def synth(%Call{}, %TypeContext{}), do: :unsupported diff --git a/lib/rustq/meta/pattern.ex b/lib/rustq/meta/pattern.ex index cab8c1f..e6b5f01 100644 --- a/lib/rustq/meta/pattern.ex +++ b/lib/rustq/meta/pattern.ex @@ -11,4 +11,24 @@ defmodule RustQ.Meta.Pattern do do: Tuple.to_list(pattern) def tuple_elements(_pattern), do: nil + + @doc "The variables a pattern binds, in order, leaving out pinned and underscored ones." + @spec variables(Macro.t()) :: [{atom(), keyword(), atom()}] + def variables(pattern) do + {_pattern, variables} = + Macro.prewalk(pattern, [], fn + {:^, _meta, _args}, variables -> + {:pinned, variables} + + {name, meta, context} = variable, variables when is_atom(name) and is_atom(context) -> + if name |> Atom.to_string() |> String.starts_with?("_"), + do: {variable, variables}, + else: {variable, [{name, meta, context} | variables]} + + other, variables -> + {other, variables} + end) + + variables |> Enum.reverse() |> Enum.uniq_by(fn {name, _meta, context} -> {name, context} end) + end end diff --git a/lib/rustq/meta/type.ex b/lib/rustq/meta/type.ex index 8d694da..dd7aa9f 100644 --- a/lib/rustq/meta/type.ex +++ b/lib/rustq/meta/type.ex @@ -632,6 +632,9 @@ defmodule RustQ.Meta.Type do }) end + # A `Vec` read from Rust is a list, as one a spec names is. + defp from_syn_path_parts(["Vec"], _name, [item], _assoc, _generic_args), do: vec(item) + defp from_syn_path_parts(parts, name, args, assoc, generic_args) do ast = %AST.TypePath{ parts: Enum.map(parts, &Identifier.atom!/1), @@ -1233,8 +1236,11 @@ defmodule RustQ.Meta.Type do type(:vec, %AST.TypeVec{inner: inner.ast}, %{inner: inner}) end + # The parsed type arguments are kept, so a map's key and value keep what their specs say. defp parse_external_type({:__aliases__, _, parts}, :t, args, aliases) do - type(:type, external_type_path(parts, args, aliases), external_type_meta(parts, :t, args)) + generics = for arg <- args, not lifetime_marker?(arg), do: parse(arg, aliases) + meta = Map.put(external_type_meta(parts, :t, args), :generics, generics) + type(:type, external_type_path(parts, args, aliases), meta) end defp parse_external_type({:__aliases__, _, parts}, function, args, aliases) do @@ -1391,6 +1397,9 @@ defmodule RustQ.Meta.Type do defp external_type_generics(arg, aliases), do: [parse(arg, aliases).ast] + defp lifetime_marker?({{:., _, [module, :lifetime]}, _, [_name]}), do: type_module?(module) + defp lifetime_marker?(_arg), do: false + defp rust_module_part(part) when is_atom(part), do: Identifier.atom!(Macro.underscore(Atom.to_string(part))) @@ -1398,6 +1407,13 @@ defmodule RustQ.Meta.Type do @doc "Type metadata for a RustQ AST type." @spec ast_type(AST.type()) :: t() + def ast_type(%AST.TypePath{parts: [:Vec], generics: [item]}), do: vec(ast_type(item)) + def ast_type(%AST.TypeVec{inner: item}), do: vec(ast_type(item)) + + def ast_type(%AST.TypeRef{inner: inner, mutable: false} = ast), + do: %{ref(ast_type(inner)) | ast: ast} + + def ast_type(%AST.TypeTuple{items: [_ | _] = items}), do: tuple(Enum.map(items, &ast_type/1)) def ast_type(ast), do: type(ast_type_kind(ast), ast) defp ast_type_kind(%AST.TypeNifResult{}), do: :nif_result diff --git a/lib/rustq/meta/typing.ex b/lib/rustq/meta/typing.ex index 0699892..0650ba8 100644 --- a/lib/rustq/meta/typing.ex +++ b/lib/rustq/meta/typing.ex @@ -149,6 +149,17 @@ defmodule RustQ.Meta.Typing do end end + # `RustQ.Meta.Desugar`'s iteration of a list is the list, and enumerating a list pairs each + # item with its index. + def synth({:__rustq_items__, _meta, [list]}, %Env{} = env), do: synth(list, env) + + def synth({:|>, _meta, [list, {:enumerate, _, []}]}, %Env{} = env) do + with %Type{} = list <- synth(list, env), + %Type{} = item <- Type.vec_inner(list) || slice_item(list) do + Type.vec(Type.tuple([RustQ.Spec.type(quote(do: RustQ.Type.usize())), item])) + end + end + def synth({:|>, _meta, [left, right]}, %Env{} = env) do case Call.pipe_remote(left, right) do {:ok, call} -> synth(call, env) @@ -163,6 +174,35 @@ defmodule RustQ.Meta.Typing do def synth({:{}, _meta, [first | _rest] = elements}, %Env{} = env) when not is_atom(first), do: synth_tuple(elements, env) + # Indexing a map reads a value, and indexing a list an item. + def synth({{:., _, [Access, :get]}, _meta, [collection, _key]}, %Env{} = env) do + with %Type{} = type <- synth(collection, env) do + case Collection.keyed(type) do + {:map, _key, value} -> declared(value, env) + _list -> Type.vec_inner(Type.ref_inner(type) || type) || Type.slice_inner(type) + end + end + end + + # A branching expression has the type of its first branch whose type is known, as `nil` and + # `[]` have none; a `case` branch reads what its pattern binds from the subject. + def synth({:case, _meta, [subject, [do: clauses]]}, %Env{} = env) when is_list(clauses) do + subject_type = synth(subject, env) + + Enum.find_value(clauses, fn {:->, _, [[pattern], body]} -> + synth(body, bind_pattern(unguarded(pattern), subject_type, env)) + end) + end + + def synth({:if, _meta, [_condition, branches]}, %Env{} = env) when is_list(branches), + do: Enum.find_value([:do, :else], &synth(Keyword.get(branches, &1), env)) + + def synth({:cond, _meta, [[do: clauses]]}, %Env{} = env) when is_list(clauses), + do: Enum.find_value(clauses, fn {:->, _, [_condition, body]} -> synth(body, env) end) + + def synth({:__block__, _meta, [_ | _] = expressions}, %Env{} = env), + do: synth(List.last(expressions), env) + # A list literal whose items all have one known type is a list of that type. def synth([first | rest], %Env{} = env) do with %Type{} = item <- synth(first, env), @@ -175,14 +215,13 @@ defmodule RustQ.Meta.Typing do # A comprehension collects its body's values into a list, or into the map or set `into:` # names, as `RustQ.Meta.Desugar` writes `Map.new/1,2` and `MapSet.new/1,2`. - def synth({:for, _meta, [{:<-, _, [pattern, collection]} | rest]}, %Env{} = env) do - {_qualifiers, options} = Enum.split_while(rest, &(not Keyword.keyword?(&1) or &1 == [])) + def synth({:for, _meta, [{:<-, _, _generator} | _rest] = args}, %Env{} = env) do + {qualifiers, options} = Enum.split_while(args, &(not Keyword.keyword?(&1) or &1 == [])) options = Enum.concat(options) - with %Type{} = list <- synth(collection, env), - %Type{} = item <- Type.vec_inner(list) || slice_item(list), + with %Env{} = bound <- Enum.reduce_while(qualifiers, env, &bind_generator/2), {:ok, body} <- Keyword.fetch(options, :do), - %Type{} = value <- synth(body, bind_pattern(pattern, item, env)) do + %Type{} = value <- synth(body, bound) do collected(Keyword.get(options, :into), value) else _unknown -> nil @@ -217,6 +256,14 @@ defmodule RustQ.Meta.Typing do when is_atom(name) and is_atom(context), do: %{env | vars: Map.put(env.vars, name, item)} + # `{:some, value}` and `{:ok, value}` bind what the option or result holds. + defp bind_pattern({:some, value}, %Type{kind: :option} = item, %Env{} = env), + do: bind_inner(value, item, env) + + defp bind_pattern({:ok, value}, %Type{kind: kind} = item, %Env{} = env) + when kind in [:result, :nif_result], + do: bind_inner(value, item, env) + defp bind_pattern( {left, right}, %Type{kind: :tuple, meta: %{elements: [left_type, right_type]}}, @@ -229,6 +276,53 @@ defmodule RustQ.Meta.Typing do defp bind_pattern(_pattern, _item, %Env{} = env), do: env + # A type named in Rust as one of the module's aliases is that alias, as `Table` read out of + # a map of tables. + defp declared(%Type{ast: %AST.TypePath{parts: [name], generics: []}} = type, %Env{} = env) do + rust_name = to_string(name) + + Enum.find_value(env.type_aliases, type, fn {_key, alias} -> + if alias.rust == rust_name, do: alias + end) + end + + defp declared(type, _env), do: type + + defp unguarded({:when, _, [pattern, _guard]}), do: pattern + defp unguarded(pattern), do: pattern + + defp bind_inner(pattern, %Type{} = wrapper, %Env{} = env) do + case Type.inner(wrapper) do + %Type{} = inner -> bind_pattern(pattern, inner, env) + nil -> env + end + end + + # Each generator binds its pattern to the items of its collection, in the variables the + # earlier ones bound; filters bind nothing. + defp bind_generator({:<-, _, [pattern, collection]}, %Env{} = env) do + with %Type{} = list <- synth(collection, env), + %Type{} = item <- Type.vec_inner(list) || slice_item(list) || keyed_item(list) do + {:cont, bind_pattern(pattern, item, env)} + else + _unknown -> {:halt, nil} + end + end + + defp bind_generator(_filter, %Env{} = env), do: {:cont, env} + + # A map yields its keys and values, and a set its items, copied where that costs nothing and + # borrowed otherwise, as `RustQ.Meta.Lower.Comprehension` iterates them. + defp keyed_item(type) do + case Collection.keyed(type) do + {:map, key, value} -> Type.tuple([lent(key), lent(value)]) + {:set, item} -> lent(item) + nil -> nil + end + end + + defp lent(type), do: if(Collection.copy?(type), do: type, else: Type.ref(type)) + defp collected(into, %Type{} = value) do case {Collection.keyed(constructed(into)), value} do {{:map, _, _}, %Type{kind: :tuple, meta: %{elements: [key, item]}}} -> @@ -327,20 +421,20 @@ defmodule RustQ.Meta.Typing do def infer_downstream_let_types(expressions, %Env{} = env, callbacks) when is_map(callbacks) do callbacks = Map.merge(default_inference_callbacks(), callbacks) inferred = Inference.infer_downstream_let_types(expressions, env.vars, callbacks) - rhs_types = non_propagating_let_rhs_types(expressions, env) + all_rhs_types = let_rhs_types(expressions, env) + rhs_types = Map.reject(all_rhs_types, fn {_name, type} -> Type.propagates?(type) end) + # A binding takes its right-hand side's type over what its uses suggest, unless it is an + # option or result those uses unwrap. inferred_types = Map.new(inferred, fn {name, type} -> - {name, inferred_binding_type(type, Map.get(rhs_types, name))} + {name, inferred_binding_type(type, Map.get(all_rhs_types, name))} end) # An option or result nothing downstream unwraps stays one, unless the binding's type is # already known. wrapped_types = - expressions - |> Enum.flat_map(&let_rhs_type(&1, env)) - |> Map.new() - |> Map.filter(fn {name, type} -> + Map.filter(all_rhs_types, fn {name, type} -> Type.propagates?(type) and not Map.has_key?(env.vars, name) end) @@ -739,22 +833,36 @@ defmodule RustQ.Meta.Typing do end end - defp inferred_binding_type(%Type{} = inferred, %Type{} = rhs) do - if propagated_rhs_for_inferred_binding?(rhs, inferred), do: inferred, else: rhs + # Elixir's integer, as a count is, takes the integer type its use needs, as a literal does. + defp inferred_binding_type(%Type{} = inferred, %Type{kind: :i64} = rhs) do + if Type.category(inferred) == :integer, + do: inferred, + else: rhs_unless_propagated(inferred, rhs) end + defp inferred_binding_type(%Type{} = inferred, %Type{} = rhs), + do: rhs_unless_propagated(inferred, rhs) + defp inferred_binding_type(inferred, _rhs), do: inferred + defp rhs_unless_propagated(inferred, rhs) do + if propagated_rhs_for_inferred_binding?(rhs, inferred), do: inferred, else: rhs + end + defp propagated_rhs_for_inferred_binding?(%Type{} = rhs, %Type{} = inferred) do Type.propagates?(rhs) and (not Type.propagates?(inferred) or Type.compatible_with_expected?(Type.inner(rhs), inferred)) end - defp non_propagating_let_rhs_types(expressions, %Env{} = env) do - expressions - |> Enum.flat_map(&let_rhs_type(&1, env)) - |> Map.new() - |> Map.reject(fn {_name, type} -> Type.propagates?(type) end) + # Each binding's type, in order, so a later right-hand side reads the earlier bindings. + defp let_rhs_types(expressions, %Env{} = env) do + {types, _env} = + Enum.reduce(expressions, {%{}, env}, fn expression, {types, env} -> + bound = Map.new(let_rhs_type(expression, env)) + {Map.merge(types, bound), %{env | vars: Map.merge(env.vars, bound)}} + end) + + types end defp let_rhs_type({:=, _meta, [{name, _pattern_meta, context}, expression]}, %Env{} = env) @@ -765,8 +873,28 @@ defmodule RustQ.Meta.Typing do end end + # A tuple of variables binds each to its element's type; a fallible right-hand side binds + # what it gives on success, as it propagates its error. + defp let_rhs_type({:=, _meta, [pattern, expression]}, %Env{} = env) do + with [_ | _] = names <- Pattern.tuple_elements(pattern), + %Type{kind: :tuple, meta: %{elements: elements}} <- + expression |> synth(env) |> success_type(), + true <- length(names) == length(elements) do + for {{name, _meta, context}, %Type{} = type} <- Enum.zip(names, elements), + is_atom(name) and is_atom(context) and name != :_, + do: {name, type} + else + _other -> [] + end + end + defp let_rhs_type(_expression, _env), do: [] + defp success_type(%Type{} = type), + do: if(Type.propagates?(type), do: Type.inner(type), else: type) + + defp success_type(type), do: type + defp coercion(%Type{} = actual, %Type{} = expected) do direct_coercion(actual, expected) || propagation_coercion(actual, expected) || @@ -828,6 +956,8 @@ defmodule RustQ.Meta.Typing do defp copy_deref_compatible?(_actual, _expected), do: false + defp copy?(%Type{kind: :alias, meta: %{target: %Type{} = target}}), do: copy?(target) + defp copy?(%Type{} = type) do case Type.category(Type.expected_value(type)) do category when category in [:number, :integer, :boolean, :enum] -> type.kind != :rust_enum diff --git a/lib/rustq/rust/ast/render.ex b/lib/rustq/rust/ast/render.ex index cdf1ded..7d63855 100644 --- a/lib/rustq/rust/ast/render.ex +++ b/lib/rustq/rust/ast/render.ex @@ -942,6 +942,7 @@ defmodule RustQ.Rust.AST.Render do defp render_method_receiver(%Cast{} = expr), do: ["(", render_expr(expr), ")"] defp render_method_receiver(%Match{} = expr), do: ["(", render_expr(expr), ")"] defp render_method_receiver(%If{} = expr), do: ["(", render_expr(expr), ")"] + defp render_method_receiver(%Range{} = expr), do: ["(", render_expr(expr), ")"] defp render_method_receiver(expr), do: render_expr(expr) defp render_cast_operand(%BinaryOp{} = expr), do: ["(", render_expr(expr), ")"] diff --git a/native/rustq_nif/src/decode.rs b/native/rustq_nif/src/decode.rs index 389dd2c..927fd8d 100644 --- a/native/rustq_nif/src/decode.rs +++ b/native/rustq_nif/src/decode.rs @@ -620,7 +620,7 @@ pub(crate) fn parse_method_call_expr( fn method_receiver_needs_grouping(receiver: &Expr) -> bool { matches!( receiver, - Expr::Binary(_) | Expr::Cast(_) | Expr::Reference(_) | Expr::Unary(_) + Expr::Binary(_) | Expr::Cast(_) | Expr::Range(_) | Expr::Reference(_) | Expr::Unary(_) ) } diff --git a/test/corpus/propagation/list_literal_slice_auto_borrow.rs b/test/corpus/propagation/list_literal_slice_auto_borrow.rs index acaf232..4c4c7f4 100644 --- a/test/corpus/propagation/list_literal_slice_auto_borrow.rs +++ b/test/corpus/propagation/list_literal_slice_auto_borrow.rs @@ -1,5 +1,5 @@ fn run(style: &mut TextStyle, family: String) -> NifResult<()> { - style.set_font_families(&vec![family]); + style.set_font_families(&[family]); Ok(()) } diff --git a/test/rustq/meta/lower/comprehension_test.exs b/test/rustq/meta/lower/comprehension_test.exs new file mode 100644 index 0000000..8773230 --- /dev/null +++ b/test/rustq/meta/lower/comprehension_test.exs @@ -0,0 +1,55 @@ +defmodule RustQ.Meta.Lower.ComprehensionTest do + use RustQ.Test, async: true + + defmodule Entries do + @moduledoc "Comprehensions over a list of tuples." + + use RustQ.Meta + + alias RustQ.Type, as: R + + @spec missing([{R.u32(), R.option(String.t())}]) :: + {[R.u32()], [{R.u32(), R.option(String.t())}]} + defrust missing(entries) do + ids = for {id, nil} <- entries, do: id + {ids, entries} + end + + @type group :: %{required(:names) => [String.t()]} + + @spec named(R.slice(group())) :: [String.t()] + defrust(named(groups), + do: for(group <- groups, name <- group.names, name != "", do: name.clone()) + ) + + @spec totals(R.slice(R.u32())) :: %{R.u32() => R.u32()} + defrust totals(values) do + for value <- values, reduce: %{} do + totals -> Map.update(totals, value, 1, fn count -> count + 1 end) + end + end + end + + test "a list of tuples still in use is iterated by reference, copying copy elements" do + source = rust_source!(Entries, :missing) + + assert source =~ ".iter()" + assert source =~ "(*element1, element2)" + refute source =~ ".cloned()" + end + + test "a list an item of an earlier generator holds is iterated by reference" do + source = rust_source!(Entries, :named) + + assert source =~ ~r/\.names\s+\.iter\(\)/ + refute source =~ ".into_iter()" + end + + test "a reduction with one clause binding the accumulator names it, typed" do + source = rust_source!(Entries, :totals) + + assert source =~ "let mut totals: HashMap = HashMap::new();" + assert source =~ "totals = {" + refute source =~ "__rustq_reduce" + end +end diff --git a/test/rustq/meta/lower/defrust_typing_test.exs b/test/rustq/meta/lower/defrust_typing_test.exs index 16a3b17..644ac9a 100644 --- a/test/rustq/meta/lower/defrust_typing_test.exs +++ b/test/rustq/meta/lower/defrust_typing_test.exs @@ -291,7 +291,7 @@ defmodule RustQ.Meta.Lower.DefrustTypingTest do source = AutoBorrowForReduceCase.__rustq_source__() - assert source =~ "let mut __rustq_reduce = &color;" + assert source =~ "let mut acc: &Color = &color;" assert source =~ "if flag { acc } else { acc }" assert RustQ.valid?(source, "auto_borrow_for_reduce.rs") end @@ -447,4 +447,57 @@ defmodule RustQ.Meta.Lower.DefrustTypingTest do assert source =~ "let mut builder = PathBuilder::new();" assert source =~ "builder.add_circle(Point::new(0.0, 0.0), 1.0, None);" end + + test "bindings keep the types their right-hand sides give" do + defmodule BorrowedOptionCase do + use RustQ.Meta + alias RustQ.Type, as: R + + @typep item :: %{required(:names) => [String.t()]} + @typep holder :: %{required(:base) => R.option(item())} + + @spec named(R.option(R.ref(item())), R.u32()) :: R.bool() + defrustp(named(base, _count), do: base != nil) + + @spec made() :: R.nif_result({holder(), R.u32()}) + defrustp(made(), do: ok({struct_literal(Holder, base: nil), 1})) + + @spec run() :: R.nif_result(R.bool()) + defrustp run() do + {holder, count} = made() + base = holder.base.as_ref() + ok(named(base, count)) + end + + @spec count(R.ref(holder())) :: R.usize() + defrustp count(holder) do + case holder.base do + {:some, item} -> length(item.names) + nil -> 0 + end + end + + @spec tallied(R.slice(R.u32())) :: HashMap.t(R.u32(), R.u32()) + defrustp tallied(values) do + tally = %{} + record(mut_ref(tally), values) + tally + end + + @spec record(R.mut_ref(HashMap.t(R.u32(), R.u32())), R.slice(R.u32())) :: R.unit() + defrustp record(tally, values) do + for value <- values do + tally.insert(deref(value), 1) + end + end + end + + source = BorrowedOptionCase.__rustq_source__() + + assert source =~ "Ok(named(base, count))" + assert source =~ "match &holder.base {" + assert source =~ "Some(item) => item.names.len()" + assert source =~ "let mut tally = HashMap::new();" + assert RustQ.valid?(source, "borrowed_option.rs") + end end diff --git a/test/rustq/meta/lower/enum_test.exs b/test/rustq/meta/lower/enum_test.exs index 23d5a3c..360a255 100644 --- a/test/rustq/meta/lower/enum_test.exs +++ b/test/rustq/meta/lower/enum_test.exs @@ -56,6 +56,9 @@ defmodule RustQ.Meta.Lower.EnumTest do @spec nonzero(R.ref(String.t())) :: R.option(R.usize()) defrust(nonzero(name), do: if(name.is_empty(), do: nil, else: some(name.len()))) + @spec copies(R.vec(R.ref(String.t()))) :: R.vec(String.t()) + defrust(copies(names), do: Enum.map(names, fn name -> name.clone() end)) + @spec greet_all(R.slice(String.t())) :: R.unit() defrust greet_all(names) do for name <- names do @@ -213,6 +216,10 @@ defmodule RustQ.Meta.Lower.EnumTest do assert rust_source!(Lists, :checked) =~ "unseen(&name, &HashSet::new())" end + test "cloning each item of a list of references is cloned()" do + assert rust_source!(Lists, :copies) =~ "names.into_iter().cloned().collect" + end + test "a loop over a slice reads each item by reference" do assert rust_source!(Lists, :greet_all) =~ "greet(name);" end @@ -220,7 +227,7 @@ defmodule RustQ.Meta.Lower.EnumTest do test "a generator over one value matches it beside the item" do source = rust_source!(Lists, :by_length) - assert source =~ ".map(|name| (name, nonzero(name)))" + assert source =~ ".map(|name| (nonzero(name), name))" refute source =~ "vec![" end diff --git a/test/rustq/meta/lower/map_set_test.exs b/test/rustq/meta/lower/map_set_test.exs index 5b8ab1c..6ea9dbb 100644 --- a/test/rustq/meta/lower/map_set_test.exs +++ b/test/rustq/meta/lower/map_set_test.exs @@ -17,6 +17,9 @@ defmodule RustQ.Meta.Lower.MapSetTest do @spec unseen(MapSet.t(String.t()), R.ref(MapSet.t(String.t()))) :: MapSet.t(String.t()) defrust(unseen(set, other), do: MapSet.difference(set, other)) + @spec flagged() :: MapSet.t(R.u32()) + defrust(flagged(), do: MapSet.new([1, 2])) + @spec count([String.t()]) :: integer() defrust count(names) do set = MapSet.new(names) @@ -24,6 +27,10 @@ defmodule RustQ.Meta.Lower.MapSetTest do end end + test "a set of listed items is built from an array of them" do + assert rust_source!(Seen, :flagged) =~ "HashSet::from([1, 2])" + end + test "MapSet.t is a HashSet" do assert rust_source!(Seen, :seen) =~ "fn seen(set: HashSet, name: String) -> HashSet" diff --git a/test/rustq/meta/lower/map_test.exs b/test/rustq/meta/lower/map_test.exs index 751ef21..7e6fbd9 100644 --- a/test/rustq/meta/lower/map_test.exs +++ b/test/rustq/meta/lower/map_test.exs @@ -14,6 +14,12 @@ defmodule RustQ.Meta.Lower.MapTest do @spec found(R.ref(%{String.t() => integer()}), R.ref(String.t())) :: R.option(integer()) defrust(found(scores, name), do: Map.get(scores, name)) + @spec unnamed(R.ref(%{String.t() => integer()})) :: integer() + defrust(unnamed(scores), do: Map.get(scores, "", 0)) + + @spec any_score(R.ref(%{String.t() => integer()})) :: R.option(integer()) + defrust(any_score(scores), do: scores |> Map.values() |> List.first()) + @spec known?(R.ref(%{String.t() => integer()}), R.ref(String.t())) :: R.bool() defrust(known?(scores, name), do: Map.has_key?(scores, name)) @@ -57,12 +63,20 @@ defmodule RustQ.Meta.Lower.MapTest do assert rust_source!(Scores, :known?) =~ "scores.contains_key(name)" end + test "takes the first value from the iterator over values" do + assert rust_source!(Scores, :any_score) =~ "scores.values().copied().next()" + end + + test "looks a string literal key up as the reference it is" do + assert rust_source!(Scores, :unnamed) =~ ~s|scores.get("").copied().unwrap_or(0)| + end + test "update/4 gives a present value to the function and stores the default otherwise" do source = rust_source!(Scores, :bump) - assert source =~ ".remove(&rustq_key)" - assert source =~ ".map(|count| count + 1)" - assert source =~ ".unwrap_or(1)" + assert source =~ "match rustq_updated.remove(&rustq_key) {" + assert source =~ "Some(count) => count + 1," + assert source =~ "None => 1," end test "an update takes the map at its last use and copies it otherwise" do diff --git a/test/rustq/meta/lower/range_test.exs b/test/rustq/meta/lower/range_test.exs new file mode 100644 index 0000000..dfe5f9f --- /dev/null +++ b/test/rustq/meta/lower/range_test.exs @@ -0,0 +1,36 @@ +defmodule RustQ.Meta.Lower.RangeTest do + use RustQ.Test, async: true + + defmodule Counts do + @moduledoc "Ranges with dynamic bounds." + + use RustQ.Meta + + alias RustQ.Type, as: R + + @spec read() :: R.nif_result(R.u32()) + defrustp(read(), do: {:ok, 3}) + + @spec doubled(R.u32()) :: [R.u32()] + defrustp(doubled(count), do: for(value <- 1..count//1, do: value * 2)) + + @spec counted(R.mut_ref(R.vec(R.u32()))) :: R.nif_result(R.unit()) + defrustp counted(values) do + count = read() + + for value <- 1..count//1 do + values.push(value) + end + + :ok + end + end + + test "a range a method is called on is parenthesized" do + assert rust_source!(Counts, :doubled) =~ "(1..=count).into_iter()" + end + + test "a fallible bound propagates its error" do + assert rust_source!(Counts, :counted) =~ "1..=count?" + end +end diff --git a/test/rustq/meta/lower/stdlib_test.exs b/test/rustq/meta/lower/stdlib_test.exs index 772d0c2..fbb251c 100644 --- a/test/rustq/meta/lower/stdlib_test.exs +++ b/test/rustq/meta/lower/stdlib_test.exs @@ -8,6 +8,272 @@ defmodule RustQ.Meta.Lower.StdlibTest do alias RustQ.Meta.Typing alias RustQ.Rust.AST + import RustQ.Test, only: [rust_source!: 2] + + defmodule Collections do + @moduledoc "Elixir's collection functions on Rust's collections, through a reference." + + use RustQ.Meta + + alias RustQ.Type, as: R + + @type state :: %{ + required(:keys) => R.vec(String.t()), + required(:index) => HashMap.t(R.u32(), R.usize()) + } + + @spec sizes(R.ref(state()), R.ref(MapSet.t(R.u32()))) :: R.usize() + defrust(sizes(state, seen), + do: length(state.keys) + map_size(state.index) + MapSet.size(seen) + ) + + @spec empty(R.ref(state())) :: R.bool() + defrust(empty(state), do: Enum.empty?(state.keys) or state.index == %{}) + + @spec named(R.ref(String.t())) :: R.bool() + defrust(named(name), do: name != "") + + @spec fresh() :: state() + defrust(fresh(), do: %{keys: [], index: %{}}) + + @spec counted(R.u32()) :: HashMap.t(R.u32(), R.usize()) + defrust(counted(id), do: %{id => 1}) + + @spec ids(R.ref(state())) :: R.vec(R.u32()) + defrust(ids(state), do: for({id, _at} <- state.index, do: id)) + + @spec numbers(R.mut_ref(R.vec(R.u32())), R.u32()) :: R.unit() + defrust numbers(values, count) do + for number <- 1..count//1 do + values.push(number) + end + end + + @spec tag(R.ref(String.t())) :: R.option(R.u32()) + defrust(tag(name), do: if(name.is_empty(), do: nil, else: some(name.len() |> cast(R.u32())))) + + @spec firsts(R.slice(String.t())) :: R.vec(R.usize()) + defrust firsts(names) do + first = + for {name, index} <- names |> Enum.with_index() |> Enum.reverse(), + {:some, tag} <- [tag(name)], + into: %{}, + do: {tag, index} + + for name <- names, {:some, tag} <- [tag(name)], do: first[tag] + end + + @spec regrouped(R.vec(String.t())) :: R.vec(String.t()) + defrust regrouped(names) do + {long, short} = Enum.split_with(names, fn name -> name.len() > 3 end) + short ++ copies(long ++ long) + end + + @spec copies(R.slice(String.t())) :: R.vec(String.t()) + defrust(copies(names), do: Enum.map(names, fn name -> name.clone() end)) + + @spec grown(R.vec(R.u32()), R.u32()) :: R.vec(R.u32()) + defrust grown(values, count) do + values = values ++ [count] + values + end + + @typep paths :: HashMap.t(R.u32(), R.vec(R.usize())) + @typep tables :: HashMap.t(R.vec(R.usize()), paths()) + + @spec indexed(R.slice(R.u32())) :: paths() + defrust indexed(keys) do + keys + |> Enum.with_index() + |> Enum.reduce(%{}, fn {key, index}, table -> + Map.update(table, key, [index], fn found -> found ++ [index] end) + end) + end + + @spec single(R.vec(R.u32())) :: R.result(R.u32(), R.usize()) + defrust single(keys) do + found = Enum.reduce(keys, [], fn key, found -> kept(key, found) end) + + case found do + [key] -> {:ok, key} + found -> {:error, length(found)} + end + end + + @spec kept(R.u32(), R.vec(R.u32())) :: R.vec(R.u32()) + defrust(kept(key, found), do: found ++ [key]) + + @spec first_value(R.slice(R.u32()), R.slice(R.usize())) :: R.u32() + defrust first_value(values, path) do + case path do + [index | _rest] -> values[index] + [] -> 0 + end + end + + @spec lookup(R.ref(tables()), R.slice(R.usize()), R.u32()) :: R.vec(R.usize()) + defrust(lookup(tables, at, key), do: Map.get(tables[at], key, [])) + + @spec widened(R.ref(MapSet.t(R.u32())), R.vec(R.u32())) :: MapSet.t(R.u32()) + defrust(widened(seen, keys), do: MapSet.union(seen, MapSet.new(keys))) + + @spec bumped(R.vec(R.u32()), R.usize()) :: R.vec(R.u32()) + defrust(bumped(values, index), do: List.replace_at(values, index, values[index] + 1)) + + @spec sorted_pairs(R.vec({R.u32(), R.u32()})) :: R.vec({R.u32(), R.u32()}) + defrust(sorted_pairs(pairs), do: Enum.sort(pairs)) + + @spec counted_len(R.mut_ref(R.vec(R.usize()))) :: R.usize() + defrust counted_len(values) do + position = length(values) + values.push(position) + position + end + + @spec sized(R.ref(paths())) :: R.usize() + defrust(sized(paths), do: map_size(paths)) + + @spec empty_size() :: R.usize() + defrust(empty_size(), do: sized(%{})) + + @spec wrapped(R.vec(R.u32())) :: R.vec(R.option(R.u32())) + defrust(wrapped(values), do: Enum.map(values, fn value -> some(value) end)) + + @spec joined(R.vec(R.u32()), R.vec(R.u32())) :: R.vec(R.u32()) + defrust(joined(left, right), do: left ++ right) + end + + defmodule Readings do + @moduledoc "Operators on values a pattern borrows." + + use RustQ.Meta + + alias RustQ.Type, as: R + + @typep kind :: :number | :flag + @typep reading :: R.enum(number: [R.f64()], flag: [R.bool()]) + + @spec kind(R.ref(reading())) :: kind() + defrust kind(reading) do + case reading do + {:number, _number} -> :number + {:flag, _flag} -> :flag + end + end + + @spec weight(R.ref(reading())) :: R.u32() + defrust weight(reading) do + case reading do + {:number, number} -> if number == 0.0, do: 0, else: 2 + {:flag, flag} -> if flag, do: 1, else: 0 + end + end + + @spec both(R.ref(reading()), R.ref(reading())) :: R.bool() + defrust both(left, right) do + case {left, right} do + {{:flag, a}, {:flag, b}} -> a and b + _ -> kind(left) == :number + end + end + + @spec held?(R.option(R.u32())) :: R.bool() + defrust(held?(value), do: value != nil) + + @spec counted(R.ref(MapSet.t(R.u32()))) :: R.u32() + defrust(counted(seen), do: depth(MapSet.size(seen))) + + @spec depth(R.u32()) :: R.u32() + defrust(depth(count), do: count + 1) + + @spec negative?(String.t()) :: R.bool() + defrust(negative?(word), do: String.downcase(String.trim(word)) in ["false", "no", "off"]) + end + + test "counts and emptiness of lists, maps, and sets are Rust's" do + assert rust_source!(Collections, :sizes) =~ + "state.keys.len() + state.index.len() + seen.len()" + + assert rust_source!(Collections, :empty) =~ + "state.keys.is_empty() || state.index.is_empty()" + + assert rust_source!(Collections, :named) =~ "!name.is_empty()" + end + + test "empty and literal maps build where a map is expected" do + assert rust_source!(Collections, :fresh) =~ "index: HashMap::new()" + assert rust_source!(Collections, :counted) =~ "HashMap::from([(id, 1)])" + end + + test "a map behind a reference is iterated by reference, copying copy keys" do + assert rust_source!(Collections, :ids) =~ + "state.index.iter().map(|(key, value)| (*key, *value))" + end + + test "a range with a step counts up between dynamic bounds" do + assert rust_source!(Collections, :numbers) =~ "for number in 1..=count {" + end + + test "a map a comprehension builds is indexed by a borrowed key" do + source = rust_source!(Collections, :firsts) + + assert source =~ ".rev()\n .map(" + refute source =~ ".collect()\n .into_iter()" + assert source =~ "first[&tag]" + end + + test "a list split or joined for a slice is borrowed" do + source = rust_source!(Collections, :regrouped) + + assert source =~ ".chain(long.iter().cloned())" + assert source =~ ".collect::>()," + end + + test "a reduction grows the accumulator it builds in place" do + source = rust_source!(Collections, :indexed) + + assert source =~ "HashMap::new()," + assert source =~ "Some(found) => {" + assert source =~ "rustq_updated.push(index);" + end + + test "a reduction's list is matched as a slice, and a slice index is copied" do + assert rust_source!(Collections, :single) =~ "match found.as_slice() {" + assert rust_source!(Collections, :single) =~ "[key] => Ok(*key)," + assert rust_source!(Collections, :first_value) =~ "values[*index]" + end + + test "a map read out of a map of maps is typed by its alias" do + assert rust_source!(Collections, :lookup) =~ + "tables[at].get(&key).cloned().unwrap_or_default()" + end + + test "a set behind a reference is copied, never updated in place" do + assert rust_source!(Collections, :widened) =~ "seen.clone()" + end + + test "replace_at computes the item, then sets it in place unless the index is past the end" do + source = rust_source!(Collections, :bumped) + + assert source =~ "let rustq_item = values[index] + 1;" + assert source =~ "if let Some(rustq_slot) = rustq_updated.get_mut(index) {" + assert source =~ "*rustq_slot = rustq_item;" + end + + test "lists of tuples sort as Elixir sorts them, and a count is the integer its use needs" do + assert rust_source!(Collections, :sorted_pairs) =~ "rustq_updated.sort();" + assert rust_source!(Collections, :counted_len) =~ "let position = values.len();" + end + + test "an empty map passed for a reference to one is borrowed" do + assert rust_source!(Collections, :empty_size) =~ "sized(&HashMap::new())" + end + + test "++ concatenates lists" do + assert rust_source!(Collections, :joined) =~ "rustq_updated.extend(right);" + assert rust_source!(Collections, :grown) =~ "values.push(count);" + end + test "normalizes remote stdlib calls and local Kernel forms" do assert {:ok, %Call{module: Enum, function: :count, args: [_values]}} = Call.normalize(quote(do: Enum.count(values))) @@ -96,6 +362,26 @@ defmodule RustQ.Meta.Lower.StdlibTest do end end + test "operators read borrowed copy values, and atoms take the enum compared with" do + assert rust_source!(Readings, :weight) =~ "*number == 0.0" + assert rust_source!(Readings, :weight) =~ "if *flag {" + assert rust_source!(Readings, :both) =~ "*a && *b" + assert rust_source!(Readings, :both) =~ "kind(left) == Kind::Number" + end + + test "comparing an option with nil asks whether it holds a value" do + assert rust_source!(Readings, :held?) =~ "value.is_some()" + end + + test "a count takes the integer type expected of it" do + assert rust_source!(Readings, :counted) =~ "depth(seen.len() as u32)" + end + + test "membership among literals asks an array, and strings change case in place" do + assert rust_source!(Readings, :negative?) =~ + ~s|["false", "no", "off"].contains(&word.trim().to_lowercase().as_str())| + end + test "rejects descending ranges instead of silently changing Elixir semantics" do assert_raise Diagnostic.Error, ~r/require ascending integer literal bounds/, fn -> Lower.quoted_body(quote(do: 3..1), nil) diff --git a/test/rustq/meta/typing_test.exs b/test/rustq/meta/typing_test.exs index ffe4fa8..1070c43 100644 --- a/test/rustq/meta/typing_test.exs +++ b/test/rustq/meta/typing_test.exs @@ -10,6 +10,44 @@ defmodule RustQ.Meta.TypingTest do %Type{kind: kind, rust: rust, ast: %AST.TypePath{parts: [String.to_atom(rust)]}} end + test "a branching expression has its first known branch's type" do + names = Type.vec(Type.parse(quote(do: String.t()), %{})) + maybe = Type.option(names) + env = %Typing.Env{vars: %{names: names, maybe: maybe}} + + assert %Type{rust: "Vec"} = + Typing.synth( + quote( + do: + case(maybe, + do: ( + :none -> [] + {:some, found} -> found + ) + ) + ), + env + ) + + assert %Type{rust: "Vec"} = + Typing.synth(quote(do: if(true, do: nil, else: names)), env) + + assert %Type{rust: "Vec"} = + Typing.synth( + quote( + do: + cond( + do: ( + names == [] -> nil + maybe == nil -> nil + true -> names + ) + ) + ), + env + ) + end + test "infers known clone and dereference results without guessing foreign clones" do string = Type.parse(quote(do: String.t()), %{}) reference = Type.parse(quote(do: R.ref(String.t())), %{})