diff --git a/CHANGELOG.md b/CHANGELOG.md index f6c3a5072daa..09ceb580794f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,9 +19,22 @@ All notable changes to this project will be documented in this file. * fixed the progress overlay of the graph view being dismissed while the layout updates deferred during a dataflow analysis were still being applied, which left the graph view showing its spinner * program options * added `ProgramOptions::add_flags` that takes the flags and parameters as vectors so that they can be assembled at runtime + * netlist traversal + * deprecated `netlist_utils::get_nets_at_pins` without a relocation: it is a per-pin lookup that `Gate::get_fan_in_net` and `get_fan_out_net` already provide, and it has no callers + * moved `get_common_inputs` from `netlist_utils` onto `NetlistTraversalDecorator` and deprecated the original + * added `NetlistTraversalDecorator::make_traversal_cache` and `get_gates` overloads that share results across calls through a `TraversalCache`. The traversal a cache answers for is sealed in at creation -- direction, match condition, stop rule and endpoint filters -- so a cache can never be consulted by a walk asking a different question, and everything that would make a cached answer depend on how a net was reached is excluded by construction + * fixed `get_next_sequential_gates`, `get_combinational_cone` and `get_next_sequential_gates_map` returning results with gates missing through a shared cache when the netlist contains a combinational cycle. Cache entries were written while a net was still being explored, and a cycle that led the walk back to such a net baked the partial answer into the entries of the nets in flight, so a later call reaching one of them through a side path was silently wrong -- which is how the Boolean influence plugin produced a wrong flip-flop dependency matrix on such netlists. An entry is now published only once its net, and any cycle it belongs to, is fully explored + * deprecated the four `netlist_utils::get_path` overloads, which despite the name return every gate of a cone rather than a path, in favour of `NetlistTraversalDecorator::get_gates` with a negated condition and `TraversalStop::at_mismatch` + * added `NetlistTraversalDecorator::get_gates`, the traversal that the other traversals of the decorator are special cases of. What separated them from one another was never what they collect but where they stop relative to it, which is now said out loud by a `TraversalStop` of `at_match`, `at_mismatch` or `never` rather than implied by a pair of booleans named one syllable apart. Direction is a `TraversalDirection` rather than a bare `bool successors` + * deprecated the three `netlist_utils::get_shortest_path` overloads in favour of `NetlistTraversalDecorator::get_shortest_path`, and `netlist_utils::get_ff_dependency_matrix` in favour of the one in the Boolean influence plugin, which also reports how strongly each flip-flop depends on another rather than only whether it does + * moved `get_gate_chain` and `get_complex_gate_chain` from `netlist_utils` onto `NetlistTraversalDecorator`, where the rest of the traversal lives and where a binding can keep the netlist alive for as long as Python refers to the gates it returns + * renamed `NetlistTraversalDecorator::get_next_combinational_gates` to `get_combinational_cone`, which is what it returns -- every combinational gate up to the sequential boundary, not a next layer of anything -- and removed the raw result-map cache parameter from it and from `get_next_sequential_gates`, whose reuse contract nothing enforced. Repeated traversals share results through a sealed `TraversalCache` instead. The Python bindings of both now also default `forbidden_pins` to an empty set like the C++ side always did + * added `NetlistTraversalDecorator::get_shortest_path` overloads that end at any gate of a module and that connect two modules, which existed only as free functions in `netlist_utils` before * gate library * fixed reloading a gate library destroying the library a netlist was built against, which silently replaced every gate type of that netlist. Gate libraries are now owned through a `shared_ptr` and outlive both the netlists and the Python handles that refer to them * Boolean functions + * added `to_string` to `SMT::QueryConfig`, `SMT::Model` and `SMT::SolverResult`, so that all four SMT types offer it the way `SMT::Constraint` already did instead of only an `operator<<` + * fixed the printed form of an `SMT::Model` starting with a stray comma, `{, A:5}` instead of `{A:5}` * sped up `BooleanFunction::compute_truth_table` by evaluating 64 rows of the table at once instead of running a symbolic execution per row, which walks and simplifies the entire node list every single time. Applies to single-bit functions of bitwise operations whose variables are all part of the truth table, everything else keeps using the previous implementation * raised the limit on the number of variables a truth table may be computed for from 10 to 20, see `BooleanFunction::MAX_TRUTH_TABLE_VARIABLES` * sped up the evaluation of Boolean functions, `BooleanFunction::operator<` compared two functions by building and comparing their reverse polish notation strings, which the symbolic state hit on every variable lookup @@ -31,6 +44,10 @@ All notable changes to this project will be documented in this file. * sped up evaluation with constant inputs by about 3x by folding the values directly instead of building a Boolean function per operation, which dominates the runtime of `compute_truth_table()` and thereby of the HAWKEYE S-box identification * added simplification rules for the word level operations, which the single-bit simplification through ABC cannot reach: extensions to the width the value already has, nested extensions and slices, slices that fall into one half of a concatenation or into either part of an extension, unsigned comparisons against zero and the maximum, equality of a value with its own negation, and single bit equalities and selections * Python bindings + * fixed the four `boolean_influence` functions that return influences per net handing out the nets without keeping the netlist alive: they return dicts keyed by net, and nothing protected a borrowed object sitting in a dict key + * added a warning, once per function and process, when a deprecated `NetlistUtils` function is called from Python, naming its replacement. `[[deprecated]]` warns whoever compiles, and a script has no compiler + * fixed the deprecated `NetlistUtils` bindings handing out gates and nets without keeping the netlist alive for as long as Python refers to them, which they keep doing until they are removed + * fixed `netlist_preprocessing.create_multi_bit_gate_modules` and `create_nets_at_unconnected_pins` handing out modules and nets without keeping the netlist alive: the `hal::borrowed()` call policy ties each returned object to the netlist that owns it, which works on a module-level function as well, as the owner is found through the wrapper the caller necessarily passed in * fixed the Python bindings handing out gates, nets, modules, endpoints and pins without tying them to the netlist that owns them, so that dropping the netlist left them pointing into freed memory. Reading 500 gates and 500 nets of a dropped netlist returned the wrong name and ID for 184 and 230 of them respectively, silently rather than by crashing * fixed the decorators storing a reference to the netlist or net they were constructed from without keeping it alive * fixed `NetlistGraph` never being freed by Python: its factories hand over ownership but it was bound with a non-owning holder, so every graph built from a netlist leaked, more than a gigabyte over 1500 graphs on a 3458 gate netlist @@ -45,7 +62,10 @@ All notable changes to this project will be documented in this file. * changed every binding that hands out a borrowed object to keep its **owner** alive rather than the object it was read from, through the new `hal::borrowed()` call policy that replaces `py::return_value_policy::reference_internal` at 241 places. The policy was only applied while a wrapper was being created, so whether an object was protected depended on which binding happened to hand it over first, and a module read from a gate was tied to that gate although the netlist is what owns it * fixed `DataContainer`, `ProjectDirectory`, `hawkeye.DetectionConfiguration`, `hawkeye.SBoxDatabase` and `dataflow.Configuration` leaking every instance created from Python, as each was bound with a holder that never frees. `SBoxDatabase.from_file` leaked 25 KB per call, and `ProjectManager.get_project_directory` leaked a copy on every call, as pybind11 copies a returned reference by default * fixed three enum values that were bound to a different value of their own enum, which made them indistinguishable from Python: `GateTypeProperty.fifo` was bound to `ram`, `module_identification.CandidateType.addition_offset` to `addition`, and `gui_extension_demo.ParameterType.Module` to `Gate` + * added `to_string` and `__str__` to `SMT.QueryConfig`, `SMT.Constraint`, `SMT.Model` and `SMT.SolverResult`, printing any of them showed an object address before * Plugins + * Boolean influence + * fixed `get_ff_dependency_matrix` dereferencing an uninitialized pointer on every call, which segfaulted before it returned anything. The cache it passes on was never initialized, and a pointer that is not null passed the callee's check for one * HAWKEYE * replaced `RegisterCandidate`, `RoundCandidate` and the free S-box functions of HAWKEYE with a single `CipherCandidate` that analyzes a candidate in place instead of copying it into a netlist of its own, so its gates and nets are the ones of the netlist under analysis and no longer have to be mapped back * added `CipherCandidate::identify_sboxes` that identifies every S-box of a candidate at once and annotates it with the outcome, grouping the variants the search produces of one and the same S-box and leaving a group as soon as one of them matches @@ -88,7 +108,7 @@ All notable changes to this project will be documented in this file. * fixed bug in pin model which must not crash when deleting a non-empty pin group * fixed bug by disallowing deletion of group comprising a single pin with same name * Build and dependencies - * added a test that checks the Python bindings never hand out a borrowed pointer without keeping its owner alive, and never give a class bound with a non-owning holder to a factory that returns a `unique_ptr`. It covers plugins kept in a repository of their own as well + * added a test that checks the Python bindings never hand out a borrowed pointer without keeping its owner alive, and never give a class bound with a non-owning holder to a factory that returns a `unique_ptr`. It covers plugins kept in a repository of their own as well, and holds free, static and submodule-level functions to the same rule as methods, which `hal::borrowed()` made fixable * updated the vendored igraph dependency from 0.10.12 to 1.0.1 and ported the graph algorithm and HAWKEYE plugins to the igraph 1.0 API * removed the tests below `tests/python_binding`, which were neither referenced by the build nor by any workflow and called API that no longer exists diff --git a/documentation/sphinx_doc/netlist_traversal_decorator.rst b/documentation/sphinx_doc/netlist_traversal_decorator.rst index 389e434012c0..8fadd95523da 100644 --- a/documentation/sphinx_doc/netlist_traversal_decorator.rst +++ b/documentation/sphinx_doc/netlist_traversal_decorator.rst @@ -4,4 +4,13 @@ Netlist Traversal Decorator .. autoclass:: hal_py.NetlistTraversalDecorator :members: - .. automethod:: __init__ \ No newline at end of file + .. automethod:: __init__ + +.. autoclass:: hal_py.TraversalDirection + :members: + +.. autoclass:: hal_py.TraversalStop + :members: + +.. autoclass:: hal_py.TraversalCache + :members: diff --git a/include/hal_core/netlist/boolean_function/types.h b/include/hal_core/netlist/boolean_function/types.h index c66c61fa3ef7..c1e9bc415765 100644 --- a/include/hal_core/netlist/boolean_function/types.h +++ b/include/hal_core/netlist/boolean_function/types.h @@ -141,6 +141,13 @@ namespace hal * @returns A reference to the output stream. */ friend std::ostream& operator<<(std::ostream& out, const QueryConfig& config); + + /** + * Translate the SMT query configuration into its string representation. + * + * @returns A string representing the SMT query configuration. + */ + std::string to_string() const; }; /** @@ -271,6 +278,13 @@ namespace hal */ friend std::ostream& operator<<(std::ostream& out, const Model& model); + /** + * Translate the SMT model into its string representation. + * + * @returns A string representing the SMT model. + */ + std::string to_string() const; + //////////////////////////////////////////////////////////////////////// // Interface //////////////////////////////////////////////////////////////////////// @@ -376,6 +390,13 @@ namespace hal */ friend std::ostream& operator<<(std::ostream& out, const SolverResult& result); + /** + * Translate the SMT result into its string representation. + * + * @returns A string representing the SMT result. + */ + std::string to_string() const; + private: /// Constructor to initialize a 'Result. SolverResult(SolverResultType _type, std::optional _model); diff --git a/include/hal_core/netlist/decorators/netlist_traversal_decorator.h b/include/hal_core/netlist/decorators/netlist_traversal_decorator.h index 69e0669c7e8e..550c2a61e629 100644 --- a/include/hal_core/netlist/decorators/netlist_traversal_decorator.h +++ b/include/hal_core/netlist/decorators/netlist_traversal_decorator.h @@ -33,6 +33,99 @@ namespace hal { + /** + * The direction in which a netlist is traversed. + * + * @ingroup decorators + */ + enum class TraversalDirection + { + forward, /**< Follow the fan-out, i.e., towards the successors of a gate. */ + backward, /**< Follow the fan-in, i.e., towards the predecessors of a gate. */ + both, /**< Follow both, and report the union of the two. */ + }; + + /** + * Where a traversal stops relative to the gates it is looking for. + * + * A traversal walks the netlist collecting the gates a filter accepts. What separates one + * traversal from another is not what it collects but where it comes to a halt, and these are the + * three ways that can be answered. + * + * @ingroup decorators + */ + enum class TraversalStop + { + /** + * Stop at a gate the filter accepts. The gates collected are the boundary of the search: they + * are reported but not traversed through, so what lies behind them is not reached. This is + * how one asks for the next flip-flops behind a cone of combinational logic. + */ + at_match, + + /** + * Stop at a gate the filter rejects. Only gates the filter accepts are traversed through, so + * the gates collected form one connected region of them. This is how one asks for the + * combinational logic between two registers. + */ + at_mismatch, + + /** + * Do not stop at a gate at all. Everything reachable is traversed and every gate the filter + * accepts is collected on the way. Bound this with a depth or with the endpoint filters, + * or it walks to the edges of the netlist. + */ + never, + }; + + /** + * A reusable store for the results of one specific traversal, handed to `NetlistTraversalDecorator::get_gates`. + * + * The traversal a cache belongs to is sealed in when it is created: the direction, the match + * condition, the stop rule and the endpoint filters all become part of the cache, and the cache + * can only ever be used for exactly that traversal. This is what makes reuse sound -- an entry + * for a net is the complete answer for that net, so it must never be consulted by a walk that + * asks a different question. + * + * Two things are excluded on purpose, because either would make the answer for a net depend on + * how the net was reached: there is no depth limit, and the endpoint filters receive no depth. + * + * The cache belongs to the netlist it was created for and must be dropped when the netlist is + * modified, as its entries are not invalidated by netlist events. + * + * @ingroup decorators + */ + class NETLIST_API TraversalCache + { + public: + TraversalCache(TraversalCache&&) = default; + TraversalCache& operator=(TraversalCache&&) = default; + TraversalCache(const TraversalCache&) = delete; + TraversalCache& operator=(const TraversalCache&) = delete; + + private: + friend class NetlistTraversalDecorator; + + TraversalCache(const Netlist* netlist, + TraversalDirection direction, + std::function match, + TraversalStop stop, + std::function exit_endpoint_filter, + std::function entry_endpoint_filter) + : m_netlist(netlist), m_direction(direction), m_match(std::move(match)), m_stop(stop), m_exit_endpoint_filter(std::move(exit_endpoint_filter)), + m_entry_endpoint_filter(std::move(entry_endpoint_filter)) + { + } + + const Netlist* m_netlist; + TraversalDirection m_direction; + std::function m_match; + TraversalStop m_stop; + std::function m_exit_endpoint_filter; + std::function m_entry_endpoint_filter; + std::unordered_map> m_store; + }; + /** * A netlist decorator that provides functionality to traverse the associated netlist without making any modifications. * @@ -48,9 +141,59 @@ namespace hal */ NetlistTraversalDecorator(const Netlist& netlist); + /** + * Traverse the netlist from the given net, collecting the gates that `match` accepts. + * + * This is the traversal that the other functions of this decorator are written in terms of. What + * distinguishes them from one another is `stop`, which says where the walk halts relative to the + * gates being looked for, see `TraversalStop`. + * + * @param[in] net - The net to start from. + * @param[in] direction - The direction to traverse in. + * @param[in] match - The condition a gate has to meet to be collected. + * @param[in] stop - Where to stop traversing, relative to the gates that `match` accepts. + * @param[in] max_depth - The maximum number of gates to traverse through, counted from 1 for the direct neighbours of the start. `0` for no limit. + * @param[in] exit_endpoint_filter - Condition that has to hold to leave a gate through a fan-in/out endpoint. + * @param[in] entry_endpoint_filter - Condition that has to hold to enter a gate through a successor/predecessor endpoint. + * @returns The gates that were collected on success, an error otherwise. + */ + Result> get_gates(const Net* net, + TraversalDirection direction, + const std::function& match, + TraversalStop stop, + u32 max_depth = 0, + const std::function& exit_endpoint_filter = nullptr, + const std::function& entry_endpoint_filter = nullptr) const; + + /** + * Traverse the netlist from the given gate, collecting the gates that `match` accepts. + * + * This is the traversal that the other functions of this decorator are written in terms of. What + * distinguishes them from one another is `stop`, which says where the walk halts relative to the + * gates being looked for, see `TraversalStop`. + * + * @param[in] gate - The gate to start from. + * @param[in] direction - The direction to traverse in. + * @param[in] match - The condition a gate has to meet to be collected. + * @param[in] stop - Where to stop traversing, relative to the gates that `match` accepts. + * @param[in] max_depth - The maximum number of gates to traverse through, counted from 1 for the direct neighbours of the start. `0` for no limit. + * @param[in] exit_endpoint_filter - Condition that has to hold to leave a gate through a fan-in/out endpoint. + * @param[in] entry_endpoint_filter - Condition that has to hold to enter a gate through a successor/predecessor endpoint. + * @returns The gates that were collected on success, an error otherwise. + */ + Result> get_gates(const Gate* gate, + TraversalDirection direction, + const std::function& match, + TraversalStop stop, + u32 max_depth = 0, + const std::function& exit_endpoint_filter = nullptr, + const std::function& entry_endpoint_filter = nullptr) const; + /** * Starting from the given net, traverse the netlist and return only the successor/predecessor gates for which the `target_gate_filter` evaluates to `true`. * Traverse over gates that do not meet the `target_gate_filter` condition. + * + * Equivalent to `get_gates` with a `TraversalStop` of `at_match`, or of `never` when `continue_on_match` is set. * Stop traversal if (1) `continue_on_match` is `false` the `target_gate_filter` evaluates to `true`, (2) the `exit_endpoint_filter` evaluates to `false` on a fan-in/out endpoint (i.e., when exiting the current gate during traversal), or (3) the `entry_endpoint_filter` evaluates to `false` on a successor/predecessor endpoint (i.e., when entering the next gate during traversal). * Both the `entry_endpoint_filter` and the `exit_endpoint_filter` may be omitted. * @@ -168,31 +311,27 @@ namespace hal * Starting from the given net, traverse the netlist and return only the next layer of sequential successor/predecessor gates. * Traverse over gates that are not sequential until a sequential gate is found. * Stop traversal at all sequential gates, but only adds those to the result that have not been reached through a pin of one of the forbidden types. - * Provide a cache to speed up traversal when calling this function multiple times on the same netlist using the same forbidden pins. - * + * For repeated calls on the same netlist, seal the traversal into a reusable cache with `make_traversal_cache` and call `get_gates` with it instead. + * * @param[in] net - Start net. * @param[in] successors - Set `true` to get successors, set `false` to get predecessors. * @param[in] forbidden_pins - Sequential gates reached through these pins will not be part of the result. Defaults to an empty set. - * @param[inout] cache - An optional cache that can be used for better performance on repeated calls. Defaults to a `nullptr`. * @returns The next sequential gates on success, an error otherwise. */ - Result> - get_next_sequential_gates(const Net* net, bool successors, const std::set& forbidden_pins = {}, std::unordered_map>* cache = nullptr) const; + Result> get_next_sequential_gates(const Net* net, bool successors, const std::set& forbidden_pins = {}) const; /** * Starting from the given gate, traverse the netlist and return only the next layer of sequential successor/predecessor gates. * Traverse over gates that are not sequential until a sequential gate is found. * Stop traversal at all sequential gates, but only adds those to the result that have not been reached through a pin of one of the forbidden types. - * Provide a cache to speed up traversal when calling this function multiple times on the same netlist using the same forbidden pins. - * + * For repeated calls on the same netlist, seal the traversal into a reusable cache with `make_traversal_cache` and call `get_gates` with it instead. + * * @param[in] gate - Start gate. * @param[in] successors - Set `true` to get successors, set `false` to get predecessors. * @param[in] forbidden_pins - Sequential gates reached through these pins will not be part of the result. Defaults to an empty set. - * @param[inout] cache - An optional cache that can be used for better performance on repeated calls. Defaults to a `nullptr`. * @returns The next sequential gates on success, an error otherwise. */ - Result> - get_next_sequential_gates(const Gate* gate, bool successors, const std::set& forbidden_pins = {}, std::unordered_map>* cache = nullptr) const; + Result> get_next_sequential_gates(const Gate* gate, bool successors, const std::set& forbidden_pins = {}) const; /** * Get the next sequential gates for all sequential gates in the netlist by traversing through remaining logic (e.g., combinational logic). @@ -206,36 +345,32 @@ namespace hal Result>> get_next_sequential_gates_map(bool successors, const std::set& forbidden_pins) const; /** - * Starting from the given net, traverse the netlist and return all combinational successor/predecessor gates. - * Continue traversal as long as further combinational gates are found and stop at gates that are not combinational. + * Starting from the given net, collect the combinational cone in the given direction, i.e., the combinational fan-out (`successors = true`) or fan-in (`successors = false`) of the net. + * Continue traversal as long as further combinational gates are found and stop at gates that are not combinational, so that the cone extends up to (but not including) the sequential boundary. * All combinational gates found during traversal are added to the result. - * Provide a cache to speed up traversal when calling this function multiple times on the same netlist. * Forbidden pins can be provided to, e.g., avoid the inclusion of logic in front of flip-flop control inputs. - * + * For repeated calls on the same netlist, seal the traversal into a reusable cache with `make_traversal_cache` and call `get_gates` with it instead. + * * @param[in] net - Start net. - * @param[in] successors - Set `true` to get successors, set `false` to get predecessors. + * @param[in] successors - Set `true` to get the fan-out cone, set `false` to get the fan-in cone. * @param[in] forbidden_pins - Traversal stops at pins of these types, i.e., gates reached through such a pin are not part of the result. Defaults to an empty set. - * @param[inout] cache - An optional cache that can be used for better performance on repeated calls. Defaults to a `nullptr`. - * @returns The next combinational gates on success, an error otherwise. + * @returns The gates of the combinational cone on success, an error otherwise. */ - Result> - get_next_combinational_gates(const Net* net, bool successors, const std::set& forbidden_pins = {}, std::unordered_map>* cache = nullptr) const; + Result> get_combinational_cone(const Net* net, bool successors, const std::set& forbidden_pins = {}) const; /** - * Starting from the given gate, traverse the netlist and return all combinational successor/predecessor gates. - * Continue traversal as long as further combinational gates are found and stop at gates that are not combinational. + * Starting from the given gate, collect the combinational cone in the given direction, i.e., the combinational fan-out (`successors = true`) or fan-in (`successors = false`) of the gate. + * Continue traversal as long as further combinational gates are found and stop at gates that are not combinational, so that the cone extends up to (but not including) the sequential boundary. * All combinational gates found during traversal are added to the result. - * Provide a cache to speed up traversal when calling this function multiple times on the same netlist. * Forbidden pins can be provided to, e.g., avoid the inclusion of logic in front of flip-flop control inputs. - * + * For repeated calls on the same netlist, seal the traversal into a reusable cache with `make_traversal_cache` and call `get_gates` with it instead. + * * @param[in] gate - Start gate. - * @param[in] successors - Set `true` to get successors, set `false` to get predecessors. + * @param[in] successors - Set `true` to get the fan-out cone, set `false` to get the fan-in cone. * @param[in] forbidden_pins - Traversal stops at pins of these types, i.e., gates reached through such a pin are not part of the result. Defaults to an empty set. - * @param[inout] cache - An optional cache that can be used for better performance on repeated calls. Defaults to a `nullptr`. - * @returns The next combinational gates on success, an error otherwise. + * @returns The gates of the combinational cone on success, an error otherwise. */ - Result> - get_next_combinational_gates(const Gate* gate, bool successors, const std::set& forbidden_pins = {}, std::unordered_map>* cache = nullptr) const; + Result> get_combinational_cone(const Gate* gate, bool successors, const std::set& forbidden_pins = {}) const; /** * Find the length of shortest path (i.e., the result set with the lowest number of gates) that connects the start gate with the end gate. @@ -273,9 +408,168 @@ namespace hal const std::function& exit_endpoint_filter = nullptr, const std::function& entry_endpoint_filter = nullptr) const; - // TODO move get_gate_chain and get_complex_gate_chain here + /** + * Find the shortest path (i.e., the result set with the lowest number of gates) that connects the start gate + * with any gate of the given module. The start gate will be the first in the result vector, the gate reached + * within the module the last. If there is no such path an empty optional is returned. If there is more than + * one path with the same length only the first one is returned. A start gate that already belongs to the + * module yields a path consisting of that gate alone. + * + * @param[in] start_gate - The gate to start from. + * @param[in] end_module - The module to connect to. Gates of its submodules count as belonging to it. + * @param[in] direction - The direction to search in. Can be PinDirection::input, PinDirection::output or PinDirection::inout to search both directions and return the shorter one. + * @param[in] exit_endpoint_filter - Filter condition that determines whether to stop traversal on a fan-in/out endpoint. + * @param[in] entry_endpoint_filter - Filter condition that determines whether to stop traversal on a successor/predecessor endpoint. + * @return An optional vector of gates that connect the start gate with the module on success, an error otherwise. + */ + Result>> get_shortest_path(const Gate* start_gate, + const Module* end_module, + const PinDirection& direction, + const std::function& exit_endpoint_filter = nullptr, + const std::function& entry_endpoint_filter = nullptr) const; + + /** + * Find every shortest path (i.e., the result sets with the lowest number of gates) that connects the start + * module with the end module. There may be more than one such path, so every path of that length is returned; + * each runs from a gate of the start module to a gate of the end module. If there is no such path an empty + * vector is returned. + * + * @param[in] start_module - The module to start from. Gates of its submodules count as belonging to it. + * @param[in] end_module - The module to connect to. Gates of its submodules count as belonging to it. + * @param[in] direction - The direction to search in. Can be PinDirection::input, PinDirection::output or PinDirection::inout to search both directions and return the shorter one. + * @param[in] exit_endpoint_filter - Filter condition that determines whether to stop traversal on a fan-in/out endpoint. + * @param[in] entry_endpoint_filter - Filter condition that determines whether to stop traversal on a successor/predecessor endpoint. + * @return A vector of the shortest paths connecting the two modules on success, an error otherwise. + */ + Result>> get_shortest_path(const Module* start_module, + const Module* end_module, + const PinDirection& direction, + const std::function& exit_endpoint_filter = nullptr, + const std::function& entry_endpoint_filter = nullptr) const; + + /** + * Find a chain of gates of the same type, starting at the given gate and following its output pins. + * + * @param[in] start_gate - The gate to start from. + * @param[in] input_pins - The input pins to follow. Defaults to all of them. + * @param[in] output_pins - The output pins to follow. Defaults to all of them. + * @param[in] filter - An optional filter a gate has to pass to be part of the chain. + * @returns The gates of the chain in order on success, an error otherwise. + */ + Result> get_gate_chain(Gate* start_gate, + const std::vector& input_pins = {}, + const std::vector& output_pins = {}, + const std::function& filter = nullptr) const; + + /** + * Find a chain of gates that repeats the given sequence of gate types, starting at the given gate. + * + * @param[in] start_gate - The gate to start from. + * @param[in] chain_types - The gate types the chain repeats, in order. + * @param[in] input_pins - The input pins to follow, per gate type. Defaults to all of them. + * @param[in] output_pins - The output pins to follow, per gate type. Defaults to all of them. + * @param[in] filter - An optional filter a gate has to pass to be part of the chain. + * @returns The gates of the chain in order on success, an error otherwise. + */ + Result> get_complex_gate_chain(Gate* start_gate, + const std::vector& chain_types, + const std::map>& input_pins = {}, + const std::map>& output_pins = {}, + const std::function& filter = nullptr) const; + + /** + * Get the nets that are inputs to at least `threshold` of the given gates. + * + * Shared inputs across a group of gates typically indicate a shared control signal, so this is + * a cheap way to test whether a set of gates belongs together. A `threshold` of `0` requires a + * net to feed every single one of the gates. Nets driven by GND or VCC gates do not count. + * + * @param[in] gates - The gates to inspect. + * @param[in] threshold - The number of gates a net has to feed. `0` to require all of them. Defaults to `0`. + * @returns The common input nets on success, an error otherwise. + */ + Result> get_common_inputs(const std::vector& gates, u32 threshold = 0) const; + + /** + * Create a cache for one specific traversal, to be handed to `get_gates` in place of the + * traversal's parameters. + * + * The direction must be `TraversalDirection::forward` or `backward`; a cache cannot hold + * both directions at once. The endpoint filters receive no depth, and there is no depth + * limit, as either would make the cached answers depend on how a net was reached. + * + * @param[in] direction - The direction to traverse in. + * @param[in] match - The condition a gate has to meet to be collected. + * @param[in] stop - Where to stop traversing, relative to the gates that `match` accepts. + * @param[in] exit_endpoint_filter - Condition that has to hold to leave a gate through a fan-in/out endpoint. + * @param[in] entry_endpoint_filter - Condition that has to hold to enter a gate through a successor/predecessor endpoint. + * @returns The cache. + */ + TraversalCache make_traversal_cache(TraversalDirection direction, + std::function match, + TraversalStop stop, + std::function exit_endpoint_filter = nullptr, + std::function entry_endpoint_filter = nullptr) const; + + /** + * Traverse the netlist from the given net, collecting the gates that the cache's traversal collects. + * + * The parameters of the walk live in the cache, see `make_traversal_cache`, and results are + * shared through it: what an earlier call worked out is not walked again. + * + * @param[in] net - The net to start from. + * @param[in] cache - The cache holding the traversal and its results. + * @returns The gates that were collected on success, an error otherwise. + */ + Result> get_gates(const Net* net, TraversalCache& cache) const; + + /** + * Traverse the netlist from the given gate, collecting the gates that the cache's traversal collects. + * + * The parameters of the walk live in the cache, see `make_traversal_cache`, and results are + * shared through it: what an earlier call worked out is not walked again. + * + * @param[in] gate - The gate to start from. + * @param[in] cache - The cache holding the traversal and its results. + * @returns The gates that were collected on success, an error otherwise. + */ + Result> get_gates(const Gate* gate, TraversalCache& cache) const; private: + /** + * The walk behind the cached traversals: memoized reachability over the nets, sharing its + * results through `store` across calls. + * + * An entry is published into `store` only once the exploration of its net is complete, with + * the nets of a cycle published together once their strongly connected component is done. + * Publishing earlier is what made the previous cache silently wrong: a cycle led the walk + * back to a net whose entry was still partial, the partial answer was taken, and the net + * being explored at the time kept a truncated entry forever. + * + * The parameters deliberately exclude everything that would make a per-net entry depend on + * how the net was reached: there is no depth limit, and the endpoint filters do not receive + * a depth. Entries already in `store` are trusted, so a store must only ever be reused with + * the same match condition, stop rule, direction and filters. + */ + Result> get_gates_memoized(const Net* start, + bool successors, + const std::function& match, + TraversalStop stop, + const std::function& exit_endpoint_filter, + const std::function& entry_endpoint_filter, + std::unordered_map>& store) const; + + /** + * The breadth-first search behind every get_shortest_path overload, stopping at the first gate the given + * condition accepts. Kept in one place so that searching for one gate and searching for any gate of a module + * cannot drift apart. + */ + Result>> get_shortest_path_to(const Gate* start_gate, + const std::function& is_target, + const PinDirection& direction, + const std::function& exit_endpoint_filter, + const std::function& entry_endpoint_filter) const; + const Netlist& m_netlist; }; } // namespace hal \ No newline at end of file diff --git a/include/hal_core/netlist/module.h b/include/hal_core/netlist/module.h index 8bace9b90ef9..3bc0ce8f5956 100644 --- a/include/hal_core/netlist/module.h +++ b/include/hal_core/netlist/module.h @@ -717,6 +717,7 @@ namespace hal Module* m_parent; std::unordered_map m_submodules_map; std::vector m_submodules; + std::unordered_map m_submodule_positions; // position of every submodule in m_submodules, for constant-time removal // pins u32 m_next_pin_id; @@ -743,6 +744,7 @@ namespace hal /* stores gates sorted by id */ std::unordered_map m_gates_map; std::vector m_gates; + std::unordered_map m_gate_positions; // position of every gate in m_gates, for constant-time removal std::unordered_set m_nets; std::unordered_set m_input_nets; diff --git a/include/hal_core/netlist/netlist.h b/include/hal_core/netlist/netlist.h index d247b1ba61fc..161080afb651 100644 --- a/include/hal_core/netlist/netlist.h +++ b/include/hal_core/netlist/netlist.h @@ -856,16 +856,19 @@ namespace hal std::unordered_map> m_modules_map; std::unordered_set m_modules_set; std::vector m_modules; + std::unordered_map m_module_positions; // position of every module in m_modules, for constant-time removal /* stores the nets */ std::unordered_map> m_nets_map; std::unordered_set m_nets_set; std::vector m_nets; + std::unordered_map m_net_positions; // position of every net in m_nets, for constant-time removal /* stores the gates */ std::unordered_map> m_gates_map; std::unordered_set m_gates_set; std::vector m_gates; + std::unordered_map m_gate_positions; // position of every gate in m_gates, for constant-time removal /* stores the groupings */ std::unordered_map> m_groupings_map; diff --git a/include/hal_core/netlist/netlist_utils.h b/include/hal_core/netlist/netlist_utils.h index bf4e507461c1..b6284c023b19 100644 --- a/include/hal_core/netlist/netlist_utils.h +++ b/include/hal_core/netlist/netlist_utils.h @@ -82,8 +82,10 @@ namespace hal * @param[in] nl - The netlist to extract the dependency matrix from. * @returns A pair consisting of std::map, which includes the mapping from the original gate * IDs to the ones in the matrix, and a std::vector, which is the ff dependency matrix + * \deprecated This function is deprecated, use `boolean_influence::get_ff_dependency_matrix` instead. */ - std::pair, std::vector>> get_ff_dependency_matrix(const Netlist* nl); + [[deprecated("Will be removed in a future version, use boolean_influence::get_ff_dependency_matrix instead.")]] std::pair, std::vector>> + get_ff_dependency_matrix(const Netlist* nl); /** * \deprecated @@ -201,8 +203,9 @@ namespace hal * @param[in] stop_properties - Stop recursion when reaching a gate of a type with one of the specified properties. * @param[inout] cache - The cache. * @returns All gates on the predecessor or successor path of the gate. + * \deprecated This function is deprecated, use `NetlistTraversalDecorator::get_gates` with a negated condition and `TraversalStop::at_mismatch` instead. */ - CORE_API std::vector get_path(const Gate* gate, bool get_successors, std::set stop_properties, std::unordered_map>& cache); + [[deprecated("Will be removed in a future version, use NetlistTraversalDecorator::get_gates instead.")]] CORE_API std::vector get_path(const Gate* gate, bool get_successors, std::set stop_properties, std::unordered_map>& cache); /** * Find all gates on the predeccessor or successor path of a gate. @@ -213,8 +216,9 @@ namespace hal * @param[in] get_successors - If `true`, the successor path is returned, otherwise the predecessor path is returned. * @param[in] stop_properties - Stop recursion when reaching a gate of a type with one of the specified properties. * @returns All gates on the predecessor or successor path of the gate. + * \deprecated This function is deprecated, use `NetlistTraversalDecorator::get_gates` with a negated condition and `TraversalStop::at_mismatch` instead. */ - CORE_API std::vector get_path(const Gate* gate, bool get_successors, std::set stop_properties); + [[deprecated("Will be removed in a future version, use NetlistTraversalDecorator::get_gates instead.")]] CORE_API std::vector get_path(const Gate* gate, bool get_successors, std::set stop_properties); /** * Find all gates on the predecessor or successor path of a net. @@ -228,8 +232,9 @@ namespace hal * @param[in] stop_properties - Stop recursion when reaching a gate of a type with one of the specified properties. * @param[inout] cache - The cache. * @returns All gates on the predecessor or successor path of the net. + * \deprecated This function is deprecated, use `NetlistTraversalDecorator::get_gates` with a negated condition and `TraversalStop::at_mismatch` instead. */ - CORE_API std::vector get_path(const Net* net, bool get_successors, std::set stop_properties, std::unordered_map>& cache); + [[deprecated("Will be removed in a future version, use NetlistTraversalDecorator::get_gates instead.")]] CORE_API std::vector get_path(const Net* net, bool get_successors, std::set stop_properties, std::unordered_map>& cache); /** * Find all gates on the predecessor or successor path of a net. @@ -239,8 +244,9 @@ namespace hal * @param[in] get_successors - If `true`, the successor path is returned, otherwise the predecessor path is returned. * @param[in] stop_properties - Stop recursion when reaching a gate of a type with one of the specified properties. * @returns All gates on the predecessor or successor path of the net. + * \deprecated This function is deprecated, use `NetlistTraversalDecorator::get_gates` with a negated condition and `TraversalStop::at_mismatch` instead. */ - CORE_API std::vector get_path(const Net* net, bool get_successors, std::set stop_properties); + [[deprecated("Will be removed in a future version, use NetlistTraversalDecorator::get_gates instead.")]] CORE_API std::vector get_path(const Net* net, bool get_successors, std::set stop_properties); /** * Get the nets that are connected to a subset of pins of the specified gate. @@ -248,8 +254,9 @@ namespace hal * @param[in] gate - The gate. * @param[in] pins - The targeted pins. * @returns A vector of nets connected to the pins. + * \deprecated This function is deprecated. Iterate the pins and use `Gate::get_fan_in_net` or `Gate::get_fan_out_net`, depending on each pin's direction. */ - CORE_API std::vector get_nets_at_pins(Gate* gate, std::vector pins); + [[deprecated("Will be removed in a future version, iterate the pins and use Gate::get_fan_in_net or Gate::get_fan_out_net instead.")]] CORE_API std::vector get_nets_at_pins(Gate* gate, std::vector pins); /** * \deprecated @@ -280,9 +287,10 @@ namespace hal * @param[in] gates - The gates. * @param[in] threshold - The threshold value, defaults to 0. * @returns The common input nets. + * \deprecated This function is deprecated, use `NetlistTraversalDecorator::get_common_inputs` instead. */ // TODO move to SubgraphNetlistDecorator - CORE_API std::vector get_common_inputs(const std::vector& gates, u32 threshold = 0); + [[deprecated("Will be removed in a future version, use NetlistTraversalDecorator::get_common_inputs instead.")]] CORE_API std::vector get_common_inputs(const std::vector& gates, u32 threshold = 0); /** * \deprecated @@ -298,43 +306,6 @@ namespace hal [[deprecated("Will be removed in a future version, use NetlistModificationDecorator::replace_gate instead.")]] CORE_API Result replace_gate(Gate* gate, GateType* target_type, std::map pin_map); - /** - * Find a sequence of identical gates that are connected via the specified input and output pins. - * The start gate may be any gate within a such a sequence, it is not required to be the first or the last gate. - * If input and/or output pins are specified, the gates must be connected through one of the input pins and/or one of the output pins. - * The optional filter is evaluated on every gate such that the result only contains gates matching the specified condition. - * - * @param[in] start_gate - The gate at which to start the chain detection. - * @param[in] input_pins - The input pins through which the gates must be connected. Defaults to an empty vector. - * @param[in] output_pins - The output pins through which the gates must be connected. Defaults to an empty vector. - * @param[in] filter - An optional filter function to be evaluated on each gate. - * @returns A vector of gates that form a chain on success, an error otherwise. - */ - CORE_API Result> get_gate_chain(Gate* start_gate, - const std::vector& input_pins = {}, - const std::vector& output_pins = {}, - const std::function& filter = nullptr); - - /** - * Find a sequence of gates (of the specified sequence of gate types) that are connected via the specified input and output pins. - * The start gate may be any gate within a such a sequence, it is not required to be the first or the last gate. - * However, the start gate must be of the first gate type within the repeating sequence. - * If input and/or output pins are specified for a gate type, the gates must be connected through one of the input pins and/or one of the output pins. - * The optional filter is evaluated on every gate such that the result only contains gates matching the specified condition. - * - * @param[in] start_gate - The gate at which to start the chain detection. - * @param[in] chain_types - The sequence of gate types that is expected to make up the gate chain. - * @param[in] input_pins - The input pins (of every gate type of the sequence) through which the gates must be connected. - * @param[in] output_pins - The output pins (of every gate type of the sequence) through which the gates must be connected. - * @param[in] filter - An optional filter function to be evaluated on each gate. - * @returns A vector of gates that form a chain on success, an error otherwise. - */ - CORE_API Result> get_complex_gate_chain(Gate* start_gate, - const std::vector& chain_types, - const std::map>& input_pins, - const std::map>& output_pins, - const std::function& filter = nullptr); - /** * Find the shortest path (i.e., the result set with the lowest number of gates) that connects the start gate with the end gate. * The gate where the search started from will be the first in the result vector, the end gate will be the last. @@ -344,8 +315,9 @@ namespace hal * @param[in] end_gate - The gate to connect to. * @param[in] search_both_directions - `true` to additionally check whether a shorter path from end to start exists, `false` otherwise. * @return A vector of gates that connect the start with end gate (possibly in reverse order). + * \deprecated This function is deprecated, use `NetlistTraversalDecorator::get_shortest_path` instead. */ - CORE_API std::vector get_shortest_path(Gate* start_gate, Gate* end_gate, bool search_both_directions = false); + [[deprecated("Will be removed in a future version, use NetlistTraversalDecorator::get_shortest_path instead.")]] CORE_API std::vector get_shortest_path(Gate* start_gate, Gate* end_gate, bool search_both_directions = false); /** * Find the shortest path (i.e., the result set with the lowest number of gates) that connects the start gate with any gate from the given module. @@ -356,8 +328,9 @@ namespace hal * @param[in] end_module - The module to connect to. * @param[in] forward_direction - `true` to search along the fan-out nets of the start gate, `false` to search along its fan-in nets. * @return A vector of gates that connect the start with end gate (possibly in reverse order). + * \deprecated This function is deprecated, use `NetlistTraversalDecorator::get_shortest_path` instead. */ - CORE_API std::vector get_shortest_path(Gate* start_gate, Module* end_module, bool forward_direction); + [[deprecated("Will be removed in a future version, use NetlistTraversalDecorator::get_shortest_path instead.")]] CORE_API std::vector get_shortest_path(Gate* start_gate, Module* end_module, bool forward_direction); /** @@ -369,7 +342,8 @@ namespace hal * @param[in] start_module - The module to start from. * @param[in] end_module - The module to connect to. * @return A vector of connecting vectors with gates that connect the start with end gate. + * \deprecated This function is deprecated, use `NetlistTraversalDecorator::get_shortest_path` instead. */ - CORE_API std::vector > get_shortest_path(Module* start_module, Module* end_module); + [[deprecated("Will be removed in a future version, use NetlistTraversalDecorator::get_shortest_path instead.")]] CORE_API std::vector > get_shortest_path(Module* start_module, Module* end_module); } // namespace netlist_utils } // namespace hal diff --git a/include/hal_core/python_bindings/python_bindings.h b/include/hal_core/python_bindings/python_bindings.h index acffb0ccab21..217eec299268 100644 --- a/include/hal_core/python_bindings/python_bindings.h +++ b/include/hal_core/python_bindings/python_bindings.h @@ -218,8 +218,10 @@ namespace hal if (py::isinstance(value)) { + // Keys as well as values: a dict from net to influence, say, borrows through its keys. for (auto item : py::reinterpret_borrow(value)) { + keep_owner_alive(call, item.first, receiver); keep_owner_alive(call, item.second, receiver); } return; diff --git a/include/hal_core/utilities/utils.h b/include/hal_core/utilities/utils.h index 1c4f1d639f84..381654feb1f8 100644 --- a/include/hal_core/utilities/utils.h +++ b/include/hal_core/utilities/utils.h @@ -37,6 +37,7 @@ #include #include #include +#include #include #include @@ -69,6 +70,49 @@ namespace hal return true; } + /** + * Append an element to an unordered vector whose element positions are tracked in a map. + * + * @param[in] vec - The unordered vector. + * @param[in] positions - The position of every element of the vector. + * @param[in] element - The element to append. + */ + template + CORE_API inline void indexed_vector_push_back(std::vector& vec, std::unordered_map& positions, T element) + { + positions[element] = static_cast(vec.size()); + vec.push_back(element); + } + + /** + * Erase an element from an unordered vector whose element positions are tracked in a map, in constant time. + * The last element takes the erased element's place, as with `unordered_vector_erase`. + * + * @param[in] vec - The unordered vector. + * @param[in] positions - The position of every element of the vector. + * @param[in] element - The element to delete. + * @returns `true` on success, `false` if the element is not in the vector. + */ + template + CORE_API inline bool indexed_vector_erase(std::vector& vec, std::unordered_map& positions, T element) + { + const auto it = positions.find(element); + if (it == positions.end()) + { + return false; + } + const u32 index = it->second; + positions.erase(it); + T last = vec.back(); + vec.pop_back(); + if (last != element) + { + vec[index] = last; + positions[last] = index; + } + return true; + } + /** * Check whether two vectors have the same content regardless of their order. * diff --git a/plugins/boolean_influence/python/python_bindings.cpp b/plugins/boolean_influence/python/python_bindings.cpp index 4e68bf97f0f6..577114e9dc62 100644 --- a/plugins/boolean_influence/python/python_bindings.cpp +++ b/plugins/boolean_influence/python/python_bindings.cpp @@ -176,6 +176,7 @@ namespace hal py::arg("gates"), py::arg("start_net"), py::arg("num_evaluations") = 32000, + borrowed(), R"( Compute the Boolean influence of each input net of a subcircuit on one of its output nets. The Boolean function of the start net is built from the given gates, translated into C code, and then compiled and executed for speed. @@ -204,6 +205,7 @@ namespace hal }, py::arg("gate"), py::arg("num_evaluations") = 32000, + borrowed(), R"( Compute the Boolean influence of each net that drives the data input of the given flip-flop. The Boolean function of the data input net is built, translated into C code, and then compiled and executed for speed. @@ -256,6 +258,7 @@ namespace hal }, py::arg("gates"), py::arg("start_net"), + borrowed(), R"( Compute the exact Boolean influence of each input net of a subcircuit on one of its output nets. In contrast to ``get_boolean_influences_of_subcircuit``, the function is evaluated on every possible input assignment instead of a random sample. @@ -282,6 +285,7 @@ namespace hal } }, py::arg("gate"), + borrowed(), R"( Compute the exact Boolean influence of each net that drives the data input of the given flip-flop. In contrast to ``get_boolean_influences_of_gate``, the function is evaluated on every possible input assignment instead of a random sample. diff --git a/plugins/boolean_influence/src/boolean_influence.cpp b/plugins/boolean_influence/src/boolean_influence.cpp index 03b04c2d4fc6..e498ee775652 100644 --- a/plugins/boolean_influence/src/boolean_influence.cpp +++ b/plugins/boolean_influence/src/boolean_influence.cpp @@ -382,7 +382,7 @@ int main(int argc, char *argv[]) { } // Extract all gates in front of the data port and iterate backwards until another flip flop is found. - const auto function_gates_res = NetlistTraversalDecorator(*(gate->get_netlist())).get_next_combinational_gates(data_net, false); + const auto function_gates_res = NetlistTraversalDecorator(*(gate->get_netlist())).get_combinational_cone(data_net, false); if (function_gates_res.is_error()) { return ERR_APPEND(function_gates_res.get_error(), @@ -589,7 +589,12 @@ int main(int argc, char *argv[]) { std::map gate_to_matrix_id; std::vector> matrix; - std::unordered_map>* cache; + // One traversal shared across every flip-flop below, so that a net is only followed once. + // The cache is sealed to exactly this walk: backwards, collecting sequential gates, + // stopping at them. + NetlistTraversalDecorator decorator(*nl); + auto cache = decorator.make_traversal_cache( + TraversalDirection::backward, [](const Gate* g) { return g->get_type()->has_property(GateTypeProperty::sequential); }, TraversalStop::at_match); u32 matrix_gates = 0; for (const auto& gate : nl->get_gates()) @@ -614,7 +619,7 @@ int main(int argc, char *argv[]) { std::vector line_of_matrix; std::set gates_to_add; - const auto next_seq_gates = NetlistTraversalDecorator(*nl).get_next_sequential_gates(gate, false, {}, cache); + const auto next_seq_gates = decorator.get_gates(gate, cache); if (next_seq_gates.is_error()) { return ERR_APPEND(next_seq_gates.get_error(), diff --git a/plugins/module_identification/src/architectures/lattice_ice40.cpp b/plugins/module_identification/src/architectures/lattice_ice40.cpp index 3db359982127..33656a9c10fa 100644 --- a/plugins/module_identification/src/architectures/lattice_ice40.cpp +++ b/plugins/module_identification/src/architectures/lattice_ice40.cpp @@ -1,4 +1,5 @@ #include "module_identification/architectures/lattice_ice40.h" +#include "hal_core/netlist/decorators/netlist_traversal_decorator.h" #include "hal_core/defines.h" #include "hal_core/netlist/decorators/subgraph_netlist_decorator.h" @@ -43,7 +44,7 @@ namespace hal const GateType* carry_type = current_gate->get_type(); // get carry chains by defining appropriate filter function - auto chain_res = netlist_utils::get_gate_chain(current_gate, {carry_type->get_pin_by_name("CI")}, {carry_type->get_pin_by_name("CO")}); + auto chain_res = NetlistTraversalDecorator(*nl).get_gate_chain(current_gate, {carry_type->get_pin_by_name("CI")}, {carry_type->get_pin_by_name("CO")}); if (chain_res.is_error()) { return std::vector>(); diff --git a/plugins/module_identification/src/architectures/xilinx_unisim.cpp b/plugins/module_identification/src/architectures/xilinx_unisim.cpp index 776708531ddb..0f1b3cda2dea 100644 --- a/plugins/module_identification/src/architectures/xilinx_unisim.cpp +++ b/plugins/module_identification/src/architectures/xilinx_unisim.cpp @@ -1,4 +1,5 @@ #include "module_identification/architectures/xilinx_unisim.h" +#include "hal_core/netlist/decorators/netlist_traversal_decorator.h" #include "hal_core/defines.h" #include "hal_core/netlist/gate.h" @@ -45,7 +46,7 @@ namespace hal const GateType* carry_type = current_gate->get_type(); // get carry chains by defining appropriate filter function - auto chain_res = netlist_utils::get_gate_chain(current_gate, {carry_type->get_pin_by_name("CI")}, {carry_type->get_pin_by_name("CO(3)")}); + auto chain_res = NetlistTraversalDecorator(*nl).get_gate_chain(current_gate, {carry_type->get_pin_by_name("CI")}, {carry_type->get_pin_by_name("CO(3)")}); if (chain_res.is_error()) { return std::vector>(); diff --git a/plugins/netlist_preprocessing/python/python_bindings.cpp b/plugins/netlist_preprocessing/python/python_bindings.cpp index 3cdfe67555c1..4bd80e35d22d 100644 --- a/plugins/netlist_preprocessing/python/python_bindings.cpp +++ b/plugins/netlist_preprocessing/python/python_bindings.cpp @@ -462,6 +462,7 @@ namespace hal }, py::arg("nl"), py::arg("concatenated_pin_groups"), + borrowed(), R"( Create modules from large gates like RAMs and DSPs with the option to concatenate multiple gate pin groups into larger consecutive pin groups. @@ -486,6 +487,7 @@ namespace hal } }, py::arg("nl"), + borrowed(), R"( Create a new net for every unconnected output pin of every gate of the netlist. The new nets are named ``HAL_UNCONNECTED_``. diff --git a/src/netlist/boolean_function/types.cpp b/src/netlist/boolean_function/types.cpp index 7c0a6739c540..456ad279242b 100644 --- a/src/netlist/boolean_function/types.cpp +++ b/src/netlist/boolean_function/types.cpp @@ -140,6 +140,13 @@ namespace hal return out; } + std::string QueryConfig::to_string() const + { + std::stringstream ss; + ss << *this; + return ss.str(); + } + Constraint::Constraint(BooleanFunction&& _constraint) : constraint(std::move(_constraint)) { } @@ -210,13 +217,20 @@ namespace hal std::ostream& operator<<(std::ostream& out, const Model& model) { auto s = std::accumulate(model.model.begin(), model.model.end(), std::string(), [](auto accumulator, auto entry) -> std::string { - return accumulator + ", " + std::get<0>(entry) + ":" + std::to_string(std::get<0>(std::get<1>(entry))); + return accumulator + (accumulator.empty() ? "" : ", ") + std::get<0>(entry) + ":" + std::to_string(std::get<0>(std::get<1>(entry))); }); out << "{" << s << "}"; return out; } + std::string Model::to_string() const + { + std::stringstream ss; + ss << *this; + return ss.str(); + } + Result Model::parse(const std::string& s, const SolverType& type) { // TODO: @@ -347,6 +361,13 @@ namespace hal out << "}"; return out; } + + std::string SolverResult::to_string() const + { + std::stringstream ss; + ss << *this; + return ss.str(); + } } // namespace SMT template<> diff --git a/src/netlist/decorators/netlist_traversal_decorator.cpp b/src/netlist/decorators/netlist_traversal_decorator.cpp index 16f794d4c8f4..fa979956e43a 100644 --- a/src/netlist/decorators/netlist_traversal_decorator.cpp +++ b/src/netlist/decorators/netlist_traversal_decorator.cpp @@ -1,20 +1,25 @@ #include "hal_core/netlist/decorators/netlist_traversal_decorator.h" +#include "hal_core/netlist/module.h" + #include "hal_core/netlist/gate.h" #include "hal_core/netlist/net.h" +#include + namespace hal { NetlistTraversalDecorator::NetlistTraversalDecorator(const Netlist& netlist) : m_netlist(netlist) { } - Result> NetlistTraversalDecorator::get_next_matching_gates(const Net* net, - bool successors, - const std::function& target_gate_filter, - bool continue_on_match, - const std::function& exit_endpoint_filter, - const std::function& entry_endpoint_filter) const + Result> NetlistTraversalDecorator::get_gates(const Net* net, + TraversalDirection direction, + const std::function& match, + TraversalStop stop, + u32 max_depth, + const std::function& exit_endpoint_filter, + const std::function& entry_endpoint_filter) const { if (net == nullptr) { @@ -26,15 +31,37 @@ namespace hal return ERR("net does not belong to netlist"); } - if (!target_gate_filter) + if (!match) { - return ERR("no target gate filter specified"); + return ERR("no match condition specified"); } - std::unordered_set visited; - std::vector stack = {net}; - std::vector previous; + if (direction == TraversalDirection::both) + { + auto res_forward = get_gates(net, TraversalDirection::forward, match, stop, max_depth, exit_endpoint_filter, entry_endpoint_filter); + if (res_forward.is_error()) + { + return res_forward; + } + + auto res_backward = get_gates(net, TraversalDirection::backward, match, stop, max_depth, exit_endpoint_filter, entry_endpoint_filter); + if (res_backward.is_error()) + { + return res_backward; + } + + auto gates = res_forward.get(); + gates.merge(res_backward.get()); + return OK(gates); + } + + const bool successors = (direction == TraversalDirection::forward); + std::set res; + std::vector stack = {net}; + std::vector previous = {}; + std::unordered_set visited; + while (!stack.empty()) { const Net* current = stack.back(); @@ -46,37 +73,51 @@ namespace hal continue; } + const u32 current_depth = previous.size() + 1; + if (max_depth != 0 && current_depth > max_depth) + { + stack.pop_back(); + continue; + } + visited.insert(current); bool added = false; for (const auto* entry_ep : successors ? current->get_destinations() : current->get_sources()) { - if (entry_endpoint_filter != nullptr && !entry_endpoint_filter(entry_ep, previous.size() + 1)) + if (entry_endpoint_filter != nullptr && !entry_endpoint_filter(entry_ep, current_depth)) { continue; } auto* gate = entry_ep->get_gate(); - if (target_gate_filter(gate)) + const bool matches = match(gate); + if (matches) { res.insert(gate); + } - if (!continue_on_match) - { - continue; - } + // Where the walk halts. A gate that ends it is still reported if it matched, it is only + // not traversed through. + if ((matches && stop == TraversalStop::at_match) || (!matches && stop == TraversalStop::at_mismatch)) + { + continue; } - for (const auto* exit_ep : successors ? gate->get_fan_out_endpoints() : gate->get_fan_in_endpoints()) + if (max_depth != 0 && current_depth == max_depth) { - const Net* exit_net = exit_ep->get_net(); + continue; + } - if (exit_endpoint_filter != nullptr && !exit_endpoint_filter(exit_ep, previous.size() + 1)) + for (const auto* exit_ep : successors ? gate->get_fan_out_endpoints() : gate->get_fan_in_endpoints()) + { + if (exit_endpoint_filter != nullptr && !exit_endpoint_filter(exit_ep, current_depth)) { continue; } + const Net* exit_net = exit_ep->get_net(); if (visited.find(exit_net) == visited.end()) { stack.push_back(exit_net); @@ -98,6 +139,84 @@ namespace hal return OK(res); } + Result> NetlistTraversalDecorator::get_gates(const Gate* gate, + TraversalDirection direction, + const std::function& match, + TraversalStop stop, + u32 max_depth, + const std::function& exit_endpoint_filter, + const std::function& entry_endpoint_filter) const + { + if (gate == nullptr) + { + return ERR("nullptr given as gate"); + } + + if (!m_netlist.is_gate_in_netlist(gate)) + { + return ERR("gate does not belong to netlist"); + } + + if (!match) + { + return ERR("no match condition specified"); + } + + // For both, the fan-out nets are walked forward and the fan-in nets backward, each purely. Passing + // `both` down to the per-net walks instead would also walk *forward* from a fan-in net, which runs + // into the cones of sibling gates sharing that input -- gates that are neither ancestors nor + // descendants of this one. + std::set res; + if (direction == TraversalDirection::forward || direction == TraversalDirection::both) + { + for (const auto* exit_ep : gate->get_fan_out_endpoints()) + { + if (exit_endpoint_filter != nullptr && !exit_endpoint_filter(exit_ep, 1)) + { + continue; + } + + auto res_net = get_gates(exit_ep->get_net(), TraversalDirection::forward, match, stop, max_depth, exit_endpoint_filter, entry_endpoint_filter); + if (res_net.is_error()) + { + return ERR_APPEND(res_net.get_error(), "cannot traverse from gate " + gate->get_name() + " with ID " + std::to_string(gate->get_id())); + } + res.merge(res_net.get()); + } + } + + if (direction == TraversalDirection::backward || direction == TraversalDirection::both) + { + for (const auto* exit_ep : gate->get_fan_in_endpoints()) + { + if (exit_endpoint_filter != nullptr && !exit_endpoint_filter(exit_ep, 1)) + { + continue; + } + + auto res_net = get_gates(exit_ep->get_net(), TraversalDirection::backward, match, stop, max_depth, exit_endpoint_filter, entry_endpoint_filter); + if (res_net.is_error()) + { + return ERR_APPEND(res_net.get_error(), "cannot traverse from gate " + gate->get_name() + " with ID " + std::to_string(gate->get_id())); + } + res.merge(res_net.get()); + } + } + + return OK(res); + } + + Result> NetlistTraversalDecorator::get_next_matching_gates(const Net* net, + bool successors, + const std::function& target_gate_filter, + bool continue_on_match, + const std::function& exit_endpoint_filter, + const std::function& entry_endpoint_filter) const + { + return get_gates(net, successors ? TraversalDirection::forward : TraversalDirection::backward, target_gate_filter, + continue_on_match ? TraversalStop::never : TraversalStop::at_match, 0, exit_endpoint_filter, entry_endpoint_filter); + } + Result> NetlistTraversalDecorator::get_next_matching_gates(const Gate* gate, bool successors, const std::function& target_gate_filter, @@ -142,87 +261,8 @@ namespace hal const std::function& exit_endpoint_filter, const std::function& entry_endpoint_filter) const { - if (net == nullptr) - { - return ERR("nullptr given as net"); - } - - if (!m_netlist.is_net_in_netlist(net)) - { - return ERR("net does not belong to netlist"); - } - - if (!target_gate_filter) - { - return ERR("no target gate filter specified"); - } - - std::unordered_set visited; - std::vector stack = {net}; - std::vector previous; - std::set res; - while (!stack.empty()) - { - const Net* current = stack.back(); - - if (!previous.empty() && current == previous.back()) - { - stack.pop_back(); - previous.pop_back(); - continue; - } - - visited.insert(current); - - bool added = false; - for (const auto* entry_ep : successors ? current->get_destinations() : current->get_sources()) - { - if (entry_endpoint_filter != nullptr && !entry_endpoint_filter(entry_ep, previous.size() + 1)) - { - continue; - } - - auto* g = entry_ep->get_gate(); - - if (target_gate_filter(g)) - { - res.insert(g); - } - else - { - if (!continue_on_mismatch) - { - continue; - } - } - - for (const auto* exit_ep : successors ? g->get_fan_out_endpoints() : g->get_fan_in_endpoints()) - { - if (exit_endpoint_filter != nullptr && !exit_endpoint_filter(exit_ep, previous.size() + 1)) - { - continue; - } - - const Net* n = exit_ep->get_net(); - if (visited.find(n) == visited.end()) - { - stack.push_back(n); - added = true; - } - } - } - - if (added) - { - previous.push_back(current); - } - else - { - stack.pop_back(); - } - } - - return OK(res); + return get_gates(net, successors ? TraversalDirection::forward : TraversalDirection::backward, target_gate_filter, + continue_on_mismatch ? TraversalStop::never : TraversalStop::at_mismatch, 0, exit_endpoint_filter, entry_endpoint_filter); } Result> NetlistTraversalDecorator::get_next_matching_gates_until(const Gate* gate, @@ -264,76 +304,8 @@ namespace hal Result> NetlistTraversalDecorator::get_next_matching_gates_until_depth(const Net* net, bool successors, u32 max_depth, const std::function& target_gate_filter) const { - if (net == nullptr) - { - return ERR("nullptr given as net"); - } - - if (!m_netlist.is_net_in_netlist(net)) - { - return ERR("net does not belong to netlist"); - } - - std::unordered_set visited; - std::vector stack = {net}; - std::vector previous; - std::set res; - while (!stack.empty()) - { - const Net* current = stack.back(); - - if (!previous.empty() && current == previous.back()) - { - stack.pop_back(); - previous.pop_back(); - continue; - } - - u32 current_depth = previous.size() + 1; - visited.insert(current); - - bool added = false; - for (const auto* entry_ep : successors ? current->get_destinations() : current->get_sources()) - { - if (max_depth != 0 && current_depth > max_depth) - { - continue; - } - - auto* g = entry_ep->get_gate(); - - if ((target_gate_filter == nullptr) || target_gate_filter(g)) - { - res.insert(g); - } - - for (const auto* exit_ep : successors ? g->get_fan_out_endpoints() : g->get_fan_in_endpoints()) - { - if (max_depth != 0 && current_depth == max_depth) - { - continue; - } - - const Net* n = exit_ep->get_net(); - if (visited.find(n) == visited.end()) - { - stack.push_back(n); - added = true; - } - } - } - - if (added) - { - previous.push_back(current); - } - else - { - stack.pop_back(); - } - } - - return OK(res); + const auto match = (target_gate_filter == nullptr) ? std::function([](const Gate*) { return true; }) : target_gate_filter; + return get_gates(net, successors ? TraversalDirection::forward : TraversalDirection::backward, match, TraversalStop::never, max_depth); } Result> @@ -363,121 +335,75 @@ namespace hal return OK(res); } - Result> - NetlistTraversalDecorator::get_next_sequential_gates(const Net* net, bool successors, const std::set& forbidden_pins, std::unordered_map>* cache) const + Result> NetlistTraversalDecorator::get_common_inputs(const std::vector& gates, u32 threshold) const { - if (net == nullptr) + // a threshold of zero means a net only counts when it is an input to every single gate + if (threshold == 0) { - return ERR("nullptr given as net"); + threshold = gates.size(); } - if (!m_netlist.is_net_in_netlist(net)) + std::map net_count; + for (Gate* gate : gates) { - return ERR("net does not belong to netlist"); - } - - std::unordered_set visited; - std::vector stack = {net}; - std::vector previous; - std::set res; - while (!stack.empty()) - { - const Net* current = stack.back(); - - if (!previous.empty() && current == previous.back()) + if (gate == nullptr) { - stack.pop_back(); - previous.pop_back(); - continue; + return ERR("nullptr given within gates"); } - visited.insert(current); - - bool added = false; - for (const auto* entry_ep : successors ? current->get_destinations() : current->get_sources()) + if (!m_netlist.is_gate_in_netlist(gate)) { - auto entry_pin = entry_ep->get_pin(); - auto* gate = entry_ep->get_gate(); + return ERR("gate " + gate->get_name() + " with ID " + std::to_string(gate->get_id()) + " does not belong to netlist"); + } - // stop traversal if gate is sequential - if (gate->get_type()->has_property(GateTypeProperty::sequential)) + for (Endpoint* pred : gate->get_predecessors()) + { + if (pred->get_gate()->is_gnd_gate() || pred->get_gate()->is_vcc_gate()) { - // stop traversal on forbidden pins - if (forbidden_pins.find(entry_pin->get_type()) != forbidden_pins.end()) - { - continue; - } - - // only add gate to result if it has not been reached through a forbidden pin (e.g., control pin) - res.insert(gate); - - // update cache - if (cache) - { - (*cache)[current].insert(gate); - for (const auto* n : previous) - { - (*cache)[n].insert(gate); - } - } + continue; } - else - { - for (const auto* exit_ep : successors ? gate->get_fan_out_endpoints() : gate->get_fan_in_endpoints()) - { - const Net* exit_net = exit_ep->get_net(); - const GatePin* exit_pin = exit_ep->get_pin(); - - // stop traversal on forbidden pins - if (forbidden_pins.find(exit_pin->get_type()) != forbidden_pins.end()) - { - continue; - } - if (cache) - { - if (const auto it = cache->find(exit_net); it != cache->end()) - { - const auto& cached_gates = std::get<1>(*it); + net_count[pred->get_net()]++; + } + } - // append cached gates to result - res.insert(cached_gates.begin(), cached_gates.end()); + std::vector common_inputs; + for (const auto& [net, count] : net_count) + { + if (count >= threshold) + { + common_inputs.push_back(net); + } + } - // update cache - (*cache)[current].insert(cached_gates.begin(), cached_gates.end()); - for (const auto* n : previous) - { - (*cache)[n].insert(cached_gates.begin(), cached_gates.end()); - } + return OK(common_inputs); + } - continue; - } - } + TraversalCache NetlistTraversalDecorator::make_traversal_cache(TraversalDirection direction, + std::function match, + TraversalStop stop, + std::function exit_endpoint_filter, + std::function entry_endpoint_filter) const + { + return TraversalCache(&m_netlist, direction, std::move(match), stop, std::move(exit_endpoint_filter), std::move(entry_endpoint_filter)); + } - if (visited.find(exit_net) == visited.end()) - { - stack.push_back(exit_net); - added = true; - } - } - } - } + Result> NetlistTraversalDecorator::get_gates(const Net* net, TraversalCache& cache) const + { + if (cache.m_netlist != &m_netlist) + { + return ERR("cache was created for a different netlist"); + } - if (added) - { - previous.push_back(current); - } - else - { - stack.pop_back(); - } + if (cache.m_direction == TraversalDirection::both) + { + return ERR("a cache cannot hold both directions at once, create one per direction"); } - return OK(res); + return get_gates_memoized(net, cache.m_direction == TraversalDirection::forward, cache.m_match, cache.m_stop, cache.m_exit_endpoint_filter, cache.m_entry_endpoint_filter, cache.m_store); } - Result> - NetlistTraversalDecorator::get_next_sequential_gates(const Gate* gate, bool successors, const std::set& forbidden_pins, std::unordered_map>* cache) const + Result> NetlistTraversalDecorator::get_gates(const Gate* gate, TraversalCache& cache) const { if (gate == nullptr) { @@ -486,174 +412,231 @@ namespace hal if (!m_netlist.is_gate_in_netlist(gate)) { - return ERR("net does not belong to netlist"); + return ERR("gate does not belong to netlist"); } - std::set res; - for (const auto* exit_ep : successors ? gate->get_fan_out_endpoints() : gate->get_fan_in_endpoints()) + if (cache.m_netlist != &m_netlist) { - const auto* exit_net = exit_ep->get_net(); - const auto* exit_pin = exit_ep->get_pin(); - - // stop traversal on forbidden pins - if (forbidden_pins.find(exit_pin->get_type()) != forbidden_pins.end()) - { - continue; - } - - if (cache) - { - if (const auto it = cache->find(exit_net); it != cache->end()) - { - const auto& cached_gates = std::get<1>(*it); - - // append cached gates to result - res.insert(cached_gates.begin(), cached_gates.end()); - - continue; - } - } - - const auto next_res = this->get_next_sequential_gates(exit_ep->get_net(), successors, forbidden_pins, cache); - if (next_res.is_error()) - { - return ERR(next_res.get_error()); - } - auto next = next_res.get(); - res.insert(next.begin(), next.end()); + return ERR("cache was created for a different netlist"); } - return OK(res); - } - Result>> NetlistTraversalDecorator::get_next_sequential_gates_map(bool successors, const std::set& forbidden_pins) const - { - std::map> seq_gate_map; - std::unordered_map> cache = {}; + if (cache.m_direction == TraversalDirection::both) + { + return ERR("a cache cannot hold both directions at once, create one per direction"); + } - for (auto* sg : m_netlist.get_gates([](const Gate* g) { return g->get_type()->has_property(GateTypeProperty::sequential); })) + std::set res; + for (const auto* exit_ep : (cache.m_direction == TraversalDirection::forward) ? gate->get_fan_out_endpoints() : gate->get_fan_in_endpoints()) { - if (const auto res = this->get_next_sequential_gates(sg, successors, forbidden_pins, &cache); res.is_ok()) + if (cache.m_exit_endpoint_filter != nullptr && !cache.m_exit_endpoint_filter(exit_ep)) { - seq_gate_map[sg] = res.get(); + continue; } - else + + auto res_net = get_gates(exit_ep->get_net(), cache); + if (res_net.is_error()) { - return ERR(res.get_error()); + return ERR_APPEND(res_net.get_error(), "cannot traverse from gate " + gate->get_name() + " with ID " + std::to_string(gate->get_id())); } + res.merge(res_net.get()); } - - return OK(std::move(seq_gate_map)); + return OK(res); } - Result> - NetlistTraversalDecorator::get_next_combinational_gates(const Net* net, bool successors, const std::set& forbidden_pins, std::unordered_map>* cache) const + Result> NetlistTraversalDecorator::get_gates_memoized(const Net* start, + bool successors, + const std::function& match, + TraversalStop stop, + const std::function& exit_endpoint_filter, + const std::function& entry_endpoint_filter, + std::unordered_map>& store) const { - if (net == nullptr) + if (start == nullptr) { return ERR("nullptr given as net"); } - if (!m_netlist.is_net_in_netlist(net)) + if (!m_netlist.is_net_in_netlist(start)) { return ERR("net does not belong to netlist"); } - std::unordered_set visited; - std::vector stack = {net}; - std::vector previous; - std::set res; - while (!stack.empty()) + if (!match) { - const Net* current = stack.back(); + return ERR("no match condition specified"); + } - if (!previous.empty() && current == previous.back()) - { - stack.pop_back(); - previous.pop_back(); - continue; - } + if (const auto it = store.find(start); it != store.end()) + { + return OK(it->second); + } - visited.insert(current); + // Tarjan's algorithm over the nets, iteratively. `collected` holds, per unpublished net, the + // gates found next to it plus everything absorbed from already published successors; when a + // strongly connected component completes, its members share the union of what they collected + // and are published together. + struct Frame + { + const Net* net; + std::vector successors; + size_t next = 0; + }; - bool added = false; - for (const auto* entry_ep : successors ? current->get_destinations() : current->get_sources()) + std::unordered_map index; + std::unordered_map lowlink; + std::unordered_map> collected; + std::vector scc_stack; + std::unordered_set on_stack; + std::vector frames; + u32 next_index = 0; + + const auto expand = [&](const Net* net) { + index[net] = lowlink[net] = next_index++; + scc_stack.push_back(net); + on_stack.insert(net); + + Frame frame{net, {}, 0}; + for (const auto* entry_ep : successors ? net->get_destinations() : net->get_sources()) { - auto* gate = entry_ep->get_gate(); - const auto* entry_pin = entry_ep->get_pin(); - if (!gate->get_type()->has_property(GateTypeProperty::combinational)) + if (entry_endpoint_filter != nullptr && !entry_endpoint_filter(entry_ep)) { - // stop traversal if not combinational continue; } - // stop traversal on forbidden pins - if (forbidden_pins.find(entry_pin->get_type()) != forbidden_pins.end()) + auto* gate = entry_ep->get_gate(); + const bool matches = match(gate); + if (matches) { - continue; + collected[net].insert(gate); } - // add to result if gate is combinational - res.insert(gate); + const bool traverse = (stop == TraversalStop::at_match) ? !matches : (stop == TraversalStop::at_mismatch) ? matches : true; + if (!traverse) + { + continue; + } - // update cache - if (cache) + for (const auto* exit_ep : successors ? gate->get_fan_out_endpoints() : gate->get_fan_in_endpoints()) { - (*cache)[current].insert(gate); - for (const auto* n : previous) + if (exit_endpoint_filter != nullptr && !exit_endpoint_filter(exit_ep)) { - (*cache)[n].insert(gate); + continue; } + + frame.successors.push_back(exit_ep->get_net()); } + } + frames.push_back(std::move(frame)); + }; + + expand(start); + while (!frames.empty()) + { + Frame& frame = frames.back(); + if (frame.next < frame.successors.size()) + { + const Net* child = frame.successors.at(frame.next++); + if (const auto it = store.find(child); it != store.end() && on_stack.find(child) == on_stack.end()) + { + // published, by an earlier call or by a component that completed within this one + collected[frame.net].insert(it->second.begin(), it->second.end()); + } + else if (index.find(child) == index.end()) + { + expand(child); + } + else if (on_stack.find(child) != on_stack.end()) + { + lowlink[frame.net] = std::min(lowlink[frame.net], index[child]); + } + } + else + { + const Net* net = frame.net; + frames.pop_back(); - for (const auto* exit_ep : successors ? gate->get_fan_out_endpoints() : gate->get_fan_in_endpoints()) + if (lowlink[net] == index[net]) { - const Net* exit_net = exit_ep->get_net(); - const GatePin* exit_pin = exit_ep->get_pin(); - - // stop traversal on forbidden pins - if (forbidden_pins.find(exit_pin->get_type()) != forbidden_pins.end()) - { - continue; - } - - if (cache) + // the component is complete: its members share one answer and are published together + std::vector members; + std::set total; + while (true) { - if (const auto it = cache->find(exit_net); it != cache->end()) + const Net* member = scc_stack.back(); + scc_stack.pop_back(); + on_stack.erase(member); + members.push_back(member); + auto& part = collected[member]; + total.insert(part.begin(), part.end()); + if (member == net) { - const auto& cached_gates = std::get<1>(*it); - - // append cached gates to result - res.insert(cached_gates.begin(), cached_gates.end()); - - continue; + break; } } + for (const Net* member : members) + { + store[member] = total; + } + } - if (visited.find(exit_net) == visited.end()) + if (!frames.empty()) + { + Frame& parent = frames.back(); + lowlink[parent.net] = std::min(lowlink[parent.net], lowlink[net]); + if (const auto it = store.find(net); it != store.end()) { - stack.push_back(exit_net); - added = true; + collected[parent.net].insert(it->second.begin(), it->second.end()); } } } + } - if (added) + return OK(store[start]); + } + + namespace + { + /// A filter that refuses any endpoint whose pin type is forbidden, or no filter at all when + /// nothing is forbidden, so that the walk skips the filter calls entirely. + std::function forbidden_pin_filter(const std::set& forbidden_pins) + { + if (forbidden_pins.empty()) { - previous.push_back(current); + return nullptr; } - else + return [forbidden_pins](const Endpoint* ep) { return forbidden_pins.find(ep->get_pin()->get_type()) == forbidden_pins.end(); }; + } + + /// The entry rule of the sequential traversal: a forbidden pin stops the walk at a sequential + /// gate entered through it, so that a flip-flop reached through its clock does not count, + /// while a combinational gate is traversed regardless of the pin it is entered through. + std::function sequential_entry_filter(const std::set& forbidden_pins) + { + if (forbidden_pins.empty()) { - stack.pop_back(); + return nullptr; } + return [forbidden_pins](const Endpoint* ep) { + return !(ep->get_gate()->get_type()->has_property(GateTypeProperty::sequential) && forbidden_pins.find(ep->get_pin()->get_type()) != forbidden_pins.end()); + }; } - return OK(res); + bool is_sequential(const Gate* gate) + { + return gate->get_type()->has_property(GateTypeProperty::sequential); + } + } // namespace + + // The two traversals below are the memoized walk with their conditions pinned. They exist by name + // because the questions they answer are asked constantly; anyone asking them repeatedly holds a + // TraversalCache and calls get_gates with it instead. + Result> NetlistTraversalDecorator::get_next_sequential_gates(const Net* net, bool successors, const std::set& forbidden_pins) const + { + std::unordered_map> local_store; + return get_gates_memoized(net, successors, is_sequential, TraversalStop::at_match, forbidden_pin_filter(forbidden_pins), sequential_entry_filter(forbidden_pins), local_store); } - Result> NetlistTraversalDecorator::get_next_combinational_gates(const Gate* gate, - bool successors, - const std::set& forbidden_pins, - std::unordered_map>* cache) const + Result> NetlistTraversalDecorator::get_next_sequential_gates(const Gate* gate, bool successors, const std::set& forbidden_pins) const { if (gate == nullptr) { @@ -662,41 +645,92 @@ namespace hal if (!m_netlist.is_gate_in_netlist(gate)) { - return ERR("net does not belong to netlist"); + return ERR("gate does not belong to netlist"); } std::set res; + std::unordered_map> local_store; + const auto exit_filter = forbidden_pin_filter(forbidden_pins); + const auto entry_filter = sequential_entry_filter(forbidden_pins); for (const auto* exit_ep : successors ? gate->get_fan_out_endpoints() : gate->get_fan_in_endpoints()) { - const auto* exit_net = exit_ep->get_net(); - const auto* exit_pin = exit_ep->get_pin(); - - // stop traversal on forbidden pins - if (forbidden_pins.find(exit_pin->get_type()) != forbidden_pins.end()) + if (forbidden_pins.find(exit_ep->get_pin()->get_type()) != forbidden_pins.end()) { continue; } - if (cache) + auto res_net = get_gates_memoized(exit_ep->get_net(), successors, is_sequential, TraversalStop::at_match, exit_filter, entry_filter, local_store); + if (res_net.is_error()) { - if (const auto it = cache->find(exit_net); it != cache->end()) - { - const auto& cached_gates = std::get<1>(*it); + return ERR_APPEND(res_net.get_error(), "cannot get next sequential gates of gate " + gate->get_name() + " with ID " + std::to_string(gate->get_id())); + } + res.merge(res_net.get()); + } + return OK(res); + } - // append cached gates to result - res.insert(cached_gates.begin(), cached_gates.end()); + Result>> NetlistTraversalDecorator::get_next_sequential_gates_map(bool successors, const std::set& forbidden_pins) const + { + std::map> seq_gate_map; + auto cache = make_traversal_cache(successors ? TraversalDirection::forward : TraversalDirection::backward, + is_sequential, + TraversalStop::at_match, + forbidden_pin_filter(forbidden_pins), + sequential_entry_filter(forbidden_pins)); - continue; - } + for (auto* sg : m_netlist.get_gates(&is_sequential)) + { + if (const auto res = this->get_gates(sg, cache); res.is_ok()) + { + seq_gate_map[sg] = res.get(); + } + else + { + return ERR(res.get_error()); } + } - const auto next_res = this->get_next_combinational_gates(exit_ep->get_net(), successors, forbidden_pins, cache); - if (next_res.is_error()) + return OK(std::move(seq_gate_map)); + } + + Result> NetlistTraversalDecorator::get_combinational_cone(const Net* net, bool successors, const std::set& forbidden_pins) const + { + const auto match = [](const Gate* g) { return g->get_type()->has_property(GateTypeProperty::combinational); }; + const auto filter = forbidden_pin_filter(forbidden_pins); + + std::unordered_map> local_store; + return get_gates_memoized(net, successors, match, TraversalStop::at_mismatch, filter, filter, local_store); + } + + Result> NetlistTraversalDecorator::get_combinational_cone(const Gate* gate, bool successors, const std::set& forbidden_pins) const + { + if (gate == nullptr) + { + return ERR("nullptr given as gate"); + } + + if (!m_netlist.is_gate_in_netlist(gate)) + { + return ERR("gate does not belong to netlist"); + } + + std::set res; + std::unordered_map> local_store; + const auto match = [](const Gate* g) { return g->get_type()->has_property(GateTypeProperty::combinational); }; + const auto filter = forbidden_pin_filter(forbidden_pins); + for (const auto* exit_ep : successors ? gate->get_fan_out_endpoints() : gate->get_fan_in_endpoints()) + { + if (forbidden_pins.find(exit_ep->get_pin()->get_type()) != forbidden_pins.end()) { - return ERR(next_res.get_error()); + continue; } - auto next = next_res.get(); - res.insert(next.begin(), next.end()); + + auto res_net = get_gates_memoized(exit_ep->get_net(), successors, match, TraversalStop::at_mismatch, filter, filter, local_store); + if (res_net.is_error()) + { + return ERR_APPEND(res_net.get_error(), "cannot get combinational cone of gate " + gate->get_name() + " with ID " + std::to_string(gate->get_id())); + } + res.merge(res_net.get()); } return OK(res); } @@ -810,11 +844,11 @@ namespace hal + std::to_string(end_gate->get_id()) + ": pin direction " + enum_to_string(direction) + " is not supported"); } - Result>> NetlistTraversalDecorator::get_shortest_path(const Gate* start_gate, - const Gate* end_gate, - const PinDirection& direction, - const std::function& exit_endpoint_filter, - const std::function& entry_endpoint_filter) const + Result>> NetlistTraversalDecorator::get_shortest_path_to(const Gate* start_gate, + const std::function& is_target, + const PinDirection& direction, + const std::function& exit_endpoint_filter, + const std::function& entry_endpoint_filter) const { const auto reconstruct_shortest_path = [](const Gate* start_gate, const Gate* end_gate, const std::unordered_map& origin_map) -> Result>> { Gate* _start_gate = start_gate->get_netlist()->get_gate_by_id(start_gate->get_id()); @@ -879,9 +913,9 @@ namespace hal } origin_map.insert({next_g, curr_g}); - if (next_g == end_gate) + if (is_target(next_g)) { - return reconstruct_shortest_path(start_gate, end_gate, origin_map); + return reconstruct_shortest_path(start_gate, next_g, origin_map); } next.push_back(next_g); @@ -903,13 +937,13 @@ namespace hal if (direction == PinDirection::inout) { - const auto res_backward = get_shortest_path(start_gate, end_gate, PinDirection::input); + const auto res_backward = get_shortest_path_to(start_gate, is_target, PinDirection::input, exit_endpoint_filter, entry_endpoint_filter); if (res_backward.is_error()) { return res_backward; } - const auto res_forward = get_shortest_path(start_gate, end_gate, PinDirection::output); + const auto res_forward = get_shortest_path_to(start_gate, is_target, PinDirection::output, exit_endpoint_filter, entry_endpoint_filter); if (res_forward.is_error()) { return res_forward; @@ -941,8 +975,386 @@ namespace hal return OK(path_forward); } - return ERR("cannot get shortest path between Gate " + start_gate->get_name() + " with ID " + std::to_string(start_gate->get_id()) + " and Gate " + end_gate->get_name() + " with ID " - + std::to_string(end_gate->get_id()) + ": pin direction " + enum_to_string(direction) + " is not supported"); + return ERR("cannot get shortest path from Gate " + start_gate->get_name() + " with ID " + std::to_string(start_gate->get_id()) + ": pin direction " + enum_to_string(direction) + + " is not supported"); + } + + Result>> NetlistTraversalDecorator::get_shortest_path(const Gate* start_gate, + const Gate* end_gate, + const PinDirection& direction, + const std::function& exit_endpoint_filter, + const std::function& entry_endpoint_filter) const + { + if (end_gate == nullptr) + { + return ERR("nullptr given as end gate"); + } + + return get_shortest_path_to( + start_gate, [end_gate](const Gate* gate) { return gate == end_gate; }, direction, exit_endpoint_filter, entry_endpoint_filter); + } + + Result>> NetlistTraversalDecorator::get_shortest_path(const Gate* start_gate, + const Module* end_module, + const PinDirection& direction, + const std::function& exit_endpoint_filter, + const std::function& entry_endpoint_filter) const + { + if (end_module == nullptr) + { + return ERR("nullptr given as end module"); + } + + const auto gates = end_module->get_gates(nullptr, true); + const std::unordered_set end_gates(gates.begin(), gates.end()); + + // A start gate that is already inside the module has arrived: the path is the gate itself. Searching from + // here instead would walk to a neighbour and report a path of two, and reporting nothing would not be + // distinguishable from the module being unreachable. + if (end_gates.find(start_gate) != end_gates.end()) + { + Gate* gate = start_gate->get_netlist()->get_gate_by_id(start_gate->get_id()); + return OK(std::optional>({gate})); + } + + return get_shortest_path_to( + start_gate, [&end_gates](const Gate* gate) { return end_gates.find(gate) != end_gates.end(); }, direction, exit_endpoint_filter, entry_endpoint_filter); + } + + Result>> NetlistTraversalDecorator::get_shortest_path(const Module* start_module, + const Module* end_module, + const PinDirection& direction, + const std::function& exit_endpoint_filter, + const std::function& entry_endpoint_filter) const + { + if (start_module == nullptr) + { + return ERR("nullptr given as start module"); + } + if (end_module == nullptr) + { + return ERR("nullptr given as end module"); + } + + // One search per gate of the start module, keeping every path that ties for the shortest. Searching from each + // gate rather than towards each one keeps the paths running from start to end, so that a caller does not have + // to know which end a path was grown from. + std::vector> shortest; + for (Gate* start_gate : start_module->get_gates(nullptr, true)) + { + const auto res = get_shortest_path(start_gate, end_module, direction, exit_endpoint_filter, entry_endpoint_filter); + if (res.is_error()) + { + return ERR_APPEND(res.get_error(), + "cannot get shortest path between Module " + start_module->get_name() + " with ID " + std::to_string(start_module->get_id()) + " and Module " + + end_module->get_name() + " with ID " + std::to_string(end_module->get_id()) + ": failed to search from Gate " + start_gate->get_name()); + } + + const auto& path = res.get(); + if (!path.has_value() || path.value().empty()) + { + continue; + } + + if (shortest.empty() || path.value().size() < shortest.front().size()) + { + shortest = {path.value()}; + } + else if (path.value().size() == shortest.front().size()) + { + shortest.push_back(path.value()); + } + } + + return OK(shortest); } + + + Result> + NetlistTraversalDecorator::get_gate_chain(Gate* start_gate, const std::vector& input_pins, const std::vector& output_pins, const std::function& filter) const + { + if (start_gate == nullptr) + { + return ERR("could not detect gate chain at start gate: start gate is a 'nullptr'"); + } + + // check filter on start gate + if (filter && !filter(start_gate)) + { + return ERR("could not detect gate chain at start gate '" + start_gate->get_name() + "' with ID " + std::to_string(start_gate->get_id()) + + ": filter evaluates to 'false' for start gate"); + } + + std::deque gate_chain = {start_gate}; + std::unordered_set visited_gates = {start_gate}; + const GateType* target_type = start_gate->get_type(); + bool found_next_gate; + + // move forward + const Gate* current_gate = start_gate; + do + { + found_next_gate = false; + + // check all eligible successors of current gate + std::vector successors = current_gate->get_successors([input_pins, output_pins, target_type, filter](const GatePin* ep_pin, Endpoint* ep) { + if (ep->get_gate()->get_type() == target_type) + { + if (output_pins.empty() || std::find(output_pins.begin(), output_pins.end(), ep_pin) != output_pins.end()) + { + if (input_pins.empty() || std::find(input_pins.begin(), input_pins.end(), ep->get_pin()) != input_pins.end()) + { + if (!filter || filter(ep->get_gate())) + { + return true; + } + } + } + } + return false; + }); + + if (successors.size() > 1) + { + log_debug("netlist_utils", + "detected more than one valid successor gate for gate '{}' with ID {} in netlist with ID {}.", + current_gate->get_name(), + current_gate->get_id(), + current_gate->get_netlist()->get_id()); + break; + } + else if (!successors.empty()) + { + Gate* suc_gate = successors.at(0)->get_gate(); + + if (visited_gates.find(suc_gate) != visited_gates.end()) + { + log_debug("netlist_utils", "detected a loop at gate with ID {}.", suc_gate->get_id()); + break; + } + + gate_chain.push_back(suc_gate); + visited_gates.insert(suc_gate); + current_gate = suc_gate; + found_next_gate = true; + } + } while (found_next_gate); + + // move backwards + current_gate = start_gate; + do + { + found_next_gate = false; + + // check all eligable predecessors of current gate + std::vector predecessors = current_gate->get_predecessors([input_pins, output_pins, target_type, filter](const GatePin* ep_pin, Endpoint* ep) { + if (ep->get_gate()->get_type() == target_type) + { + if (input_pins.empty() || std::find(input_pins.begin(), input_pins.end(), ep_pin) != input_pins.end()) + { + if (output_pins.empty() || std::find(output_pins.begin(), output_pins.end(), ep->get_pin()) != output_pins.end()) + { + if (!filter || filter(ep->get_gate())) + { + return true; + } + } + } + } + return false; + }); + + if (predecessors.size() > 1) + { + log_debug("netlist_utils", + "detected more than one valid predecessor gate for gate '{}' with ID {} in netlist with ID {}.", + current_gate->get_name(), + current_gate->get_id(), + current_gate->get_netlist()->get_id()); + break; + } + else if (!predecessors.empty()) + { + Gate* pred_gate = predecessors.at(0)->get_gate(); + + if (visited_gates.find(pred_gate) != visited_gates.end()) + { + log_debug("netlist_utils", "detected a loop at gate with ID {}.", pred_gate->get_id()); + break; + } + + gate_chain.push_front(pred_gate); + visited_gates.insert(pred_gate); + current_gate = pred_gate; + found_next_gate = true; + log_debug("netlist_utils", "found predecessor gate with ID {}.", pred_gate->get_id()); + } + } while (found_next_gate); + + return OK(std::vector(gate_chain.begin(), gate_chain.end())); + } + + Result> NetlistTraversalDecorator::get_complex_gate_chain(Gate* start_gate, + const std::vector& chain_types, + const std::map>& input_pins, + const std::map>& output_pins, + const std::function& filter) const + { + if (start_gate == nullptr) + { + return ERR("could not detect gate chain at start gate: start gate is a 'nullptr'"); + } + if (chain_types.size() < 2) + { + return ERR("could not detect gate chain at start gate: 'chain_types' comprises less than two target gate types"); + } + if (start_gate->get_type() != chain_types.at(0)) + { + return ERR("could not detect gate chain at start gate '" + start_gate->get_name() + "' with ID " + std::to_string(start_gate->get_id()) + ": start gate is not of type '" + + chain_types.front()->get_name() + "'"); + } + if (filter && !filter(start_gate)) + { + return ERR("could not detect gate chain at start gate '" + start_gate->get_name() + "' with ID " + std::to_string(start_gate->get_id()) + + ": filter evaluates to 'false' for start gate"); + } + + std::deque gate_chain = {start_gate}; + std::unordered_set visited_gates; + + u32 last_index = 0; + u32 current_index = (last_index + 1) % chain_types.size(); + + // move forward + bool found_next_gate; + const Gate* current_gate = start_gate; + do + { + found_next_gate = false; + + // check all successors of current gate + GateType* target_type = chain_types.at(current_index); + const std::vector& inputs = input_pins.at(target_type); + const std::vector& outputs = output_pins.at(chain_types.at(last_index)); + std::vector successors = current_gate->get_successors([target_type, inputs, outputs, filter](const GatePin* ep_pin, Endpoint* ep) { + if (ep->get_gate()->get_type() == target_type) + { + if (outputs.empty() || std::find(outputs.begin(), outputs.end(), ep_pin) != outputs.end()) + { + if (inputs.empty() || std::find(inputs.begin(), inputs.end(), ep->get_pin()) != inputs.end()) + { + if (!filter || filter(ep->get_gate())) + { + return true; + } + } + } + } + return false; + }); + + if (successors.size() > 1) + { + log_debug("netlist_utils", + "detected more than one valid successor gate for gate '{}' with ID {} in netlist with ID {}.", + current_gate->get_name(), + current_gate->get_id(), + current_gate->get_netlist()->get_id()); + break; + } + else if (!successors.empty()) + { + Gate* suc_gate = successors.at(0)->get_gate(); + + if (visited_gates.find(suc_gate) != visited_gates.end()) + { + log_debug("netlist_utils", "detected a loop at gate with ID {}.", suc_gate->get_id()); + break; + } + + gate_chain.push_back(suc_gate); + visited_gates.insert(suc_gate); + current_gate = suc_gate; + last_index = current_index; + current_index = (current_index + 1) % chain_types.size(); + found_next_gate = true; + } + } while (found_next_gate); + + // remove partial sequences at the end of the chain + while (current_index != 0) + { + gate_chain.pop_back(); + current_index--; + } + + current_gate = start_gate; + last_index = 0; + current_index = chain_types.size() - 1; + + // move backwards + do + { + found_next_gate = false; + + // check all predecessors of current gate + GateType* target_type = chain_types.at(current_index); + const std::vector& inputs = input_pins.at(chain_types.at(last_index)); + const std::vector& outputs = output_pins.at(target_type); + std::vector predecessors = current_gate->get_predecessors([target_type, inputs, outputs, filter](const GatePin* ep_pin, Endpoint* ep) { + if (ep->get_gate()->get_type() == target_type) + { + if (inputs.empty() || std::find(inputs.begin(), inputs.end(), ep_pin) != inputs.end()) + { + if (outputs.empty() || std::find(outputs.begin(), outputs.end(), ep->get_pin()) != outputs.end()) + { + if (!filter || filter(ep->get_gate())) + { + return true; + } + } + } + } + return false; + }); + + if (predecessors.size() > 1) + { + log_debug("netlist_utils", + "detected more than one valid predecessor gate for gate '{}' with ID {} in netlist with ID {}.", + current_gate->get_name(), + current_gate->get_id(), + current_gate->get_netlist()->get_id()); + break; + } + else if (!predecessors.empty()) + { + Gate* pred_gate = predecessors.at(0)->get_gate(); + + if (visited_gates.find(pred_gate) != visited_gates.end()) + { + log_debug("netlist_utils", "detected a loop at gate with ID {}.", pred_gate->get_id()); + break; + } + + gate_chain.push_front(pred_gate); + visited_gates.insert(pred_gate); + current_gate = pred_gate; + last_index = current_index; + current_index = (current_index == 0) ? chain_types.size() - 1 : current_index - 1; + found_next_gate = true; + } + } while (found_next_gate); + + // remove partial sequences at the beginning of the chain + while (last_index != 0) + { + gate_chain.pop_front(); + last_index--; + } + + return OK(std::vector(gate_chain.begin(), gate_chain.end())); + } + } // namespace hal \ No newline at end of file diff --git a/src/netlist/module.cpp b/src/netlist/module.cpp index 95ffc2899a2e..2bb103b5be34 100644 --- a/src/netlist/module.cpp +++ b/src/netlist/module.cpp @@ -1,4 +1,5 @@ #include "hal_core/netlist/module.h" +#include "hal_core/utilities/utils.h" #include "hal_core/netlist/event_system/event_handler.h" #include "hal_core/netlist/gate.h" @@ -199,26 +200,28 @@ namespace hal new_parent->set_parent_module(m_parent); } - m_parent->m_submodules_map.erase(m_id); - m_parent->m_submodules.erase(std::find(m_parent->m_submodules.begin(), m_parent->m_submodules.end(), this)); + // detach completely before the old parent re-checks its nets: is_parent_module_of() walks up the parent + // chain, so the moved subtree must already read as external to the old parent + Module* old_parent = m_parent; + old_parent->m_submodules_map.erase(m_id); + utils::indexed_vector_erase(old_parent->m_submodules, old_parent->m_submodule_positions, this); + m_parent = new_parent; if (m_internal_manager->m_net_checks_enabled) { for (Net* net : get_nets(nullptr, true)) { - if (auto res = m_parent->check_net(net, true); res.is_error()) + if (auto res = old_parent->check_net(net, true); res.is_error()) { log_error("module", "{}", res.get_error().get()); } } } - m_event_handler->notify(ModuleEvent::event::submodule_removed, m_parent, m_id); - - m_parent = new_parent; + m_event_handler->notify(ModuleEvent::event::submodule_removed, old_parent, m_id); m_parent->m_submodules_map[m_id] = this; - m_parent->m_submodules.push_back(this); + utils::indexed_vector_push_back(m_parent->m_submodules, m_parent->m_submodule_positions, this); if (m_internal_manager->m_net_checks_enabled) { @@ -243,18 +246,22 @@ namespace hal { return false; } - for (auto sm : m_submodules) + // walk up the parent chain of the given module rather than down this module's subtree: the chain is + // at most as long as the hierarchy is deep, whereas the subtree of a module near the root can hold most + // of the netlist, and this query runs once per endpoint when module nets are recomputed + const Module* parent = module->m_parent; + if (!recursive) { - if (sm == module) - { - return true; - } - else if (recursive && sm->is_parent_module_of(module, true)) + return parent == this; + } + while (parent != nullptr) + { + if (parent == this) { return true; } + parent = parent->m_parent; } - return false; } diff --git a/src/netlist/netlist_internal_manager.cpp b/src/netlist/netlist_internal_manager.cpp index c2646cb0b8db..36e5c96d291d 100644 --- a/src/netlist/netlist_internal_manager.cpp +++ b/src/netlist/netlist_internal_manager.cpp @@ -321,10 +321,10 @@ namespace hal m_netlist->m_gates_map[id] = std::move(new_gate); m_netlist->m_gates_set.insert(raw); - m_netlist->m_gates.push_back(raw); + utils::indexed_vector_push_back(m_netlist->m_gates, m_netlist->m_gate_positions, raw); m_netlist->m_top_module->m_gates_map[id] = raw; - m_netlist->m_top_module->m_gates.push_back(raw); + utils::indexed_vector_push_back(m_netlist->m_top_module->m_gates, m_netlist->m_top_module->m_gate_positions, raw); // notify m_event_handler->notify(ModuleEvent::event::gate_assigned, m_netlist->m_top_module, id); @@ -368,13 +368,13 @@ namespace hal // remove gate from modules gate->m_module->m_gates_map.erase(gate->m_module->m_gates_map.find(gate->get_id())); - utils::unordered_vector_erase(gate->m_module->m_gates, gate); + utils::indexed_vector_erase(gate->m_module->m_gates, gate->m_module->m_gate_positions, gate); auto it = m_netlist->m_gates_map.find(gate->get_id()); auto ptr = std::move(it->second); m_netlist->m_gates_map.erase(it); m_netlist->m_gates_set.erase(gate); - utils::unordered_vector_erase(m_netlist->m_gates, gate); + utils::indexed_vector_erase(m_netlist->m_gates, m_netlist->m_gate_positions, gate); // free ids m_netlist->m_free_gate_ids.insert(gate->get_id()); @@ -427,7 +427,7 @@ namespace hal auto raw = new_net.get(); m_netlist->m_nets_map[id] = std::move(new_net); m_netlist->m_nets_set.insert(raw); - m_netlist->m_nets.push_back(raw); + utils::indexed_vector_push_back(m_netlist->m_nets, m_netlist->m_net_positions, raw); // notify m_event_handler->notify(NetEvent::event::created, raw); @@ -475,7 +475,7 @@ namespace hal auto ptr = std::move(it->second); m_netlist->m_nets_map.erase(it); m_netlist->m_nets_set.erase(net); - utils::unordered_vector_erase(m_netlist->m_nets, net); + utils::indexed_vector_erase(m_netlist->m_nets, m_netlist->m_net_positions, net); m_netlist->m_free_net_ids.insert(net->get_id()); m_netlist->m_used_net_ids.erase(net->get_id()); @@ -804,12 +804,12 @@ namespace hal auto raw = m.get(); m_netlist->m_modules_map[id] = std::move(m); m_netlist->m_modules_set.insert(raw); - m_netlist->m_modules.push_back(raw); + utils::indexed_vector_push_back(m_netlist->m_modules, m_netlist->m_module_positions, raw); if (parent != nullptr) { parent->m_submodules_map[id] = raw; - parent->m_submodules.push_back(raw); + utils::indexed_vector_push_back(parent->m_submodules, parent->m_submodule_positions, raw); } m_event_handler->notify(ModuleEvent::event::created, raw); @@ -850,7 +850,7 @@ namespace hal for (auto sm : to_remove->m_submodules) { to_remove->m_parent->m_submodules_map[sm->get_id()] = sm; - to_remove->m_parent->m_submodules.push_back(sm); + utils::indexed_vector_push_back(to_remove->m_parent->m_submodules, to_remove->m_parent->m_submodule_positions, sm); m_event_handler->notify(ModuleEvent::event::submodule_removed, sm->get_parent_module(), sm->get_id()); @@ -862,14 +862,14 @@ namespace hal // remove module from parent to_remove->m_parent->m_submodules_map.erase(to_remove->get_id()); - utils::unordered_vector_erase(to_remove->m_parent->m_submodules, to_remove); + utils::indexed_vector_erase(to_remove->m_parent->m_submodules, to_remove->m_parent->m_submodule_positions, to_remove); m_event_handler->notify(ModuleEvent::event::submodule_removed, to_remove->m_parent, to_remove->get_id()); auto it = m_netlist->m_modules_map.find(to_remove->get_id()); auto ptr = std::move(it->second); m_netlist->m_modules_map.erase(it); m_netlist->m_modules_set.erase(to_remove); - utils::unordered_vector_erase(m_netlist->m_modules, to_remove); + utils::indexed_vector_erase(m_netlist->m_modules, m_netlist->m_module_positions, to_remove); m_netlist->m_free_module_ids.insert(to_remove->get_id()); m_netlist->m_used_module_ids.erase(to_remove->get_id()); @@ -946,11 +946,11 @@ namespace hal assert(it != prev_mod->m_gates_map.end()); prev_mod->m_gates_map.erase(it); - utils::unordered_vector_erase(prev_mod->m_gates, g); + utils::indexed_vector_erase(prev_mod->m_gates, prev_mod->m_gate_positions, g); // move gate to new module module->m_gates_map[g->get_id()] = g; - module->m_gates.push_back(g); + utils::indexed_vector_push_back(module->m_gates, module->m_gate_positions, g); g->m_module = module; // collect affected nets diff --git a/src/netlist/netlist_utils.cpp b/src/netlist/netlist_utils.cpp index eb83d09649da..f37d841cf3e6 100644 --- a/src/netlist/netlist_utils.cpp +++ b/src/netlist/netlist_utils.cpp @@ -906,290 +906,6 @@ namespace hal } } - Result> - get_gate_chain(Gate* start_gate, const std::vector& input_pins, const std::vector& output_pins, const std::function& filter) - { - if (start_gate == nullptr) - { - return ERR("could not detect gate chain at start gate: start gate is a 'nullptr'"); - } - - // check filter on start gate - if (filter && !filter(start_gate)) - { - return ERR("could not detect gate chain at start gate '" + start_gate->get_name() + "' with ID " + std::to_string(start_gate->get_id()) - + ": filter evaluates to 'false' for start gate"); - } - - std::deque gate_chain = {start_gate}; - std::unordered_set visited_gates = {start_gate}; - const GateType* target_type = start_gate->get_type(); - bool found_next_gate; - - // move forward - const Gate* current_gate = start_gate; - do - { - found_next_gate = false; - - // check all eligible successors of current gate - std::vector successors = current_gate->get_successors([input_pins, output_pins, target_type, filter](const GatePin* ep_pin, Endpoint* ep) { - if (ep->get_gate()->get_type() == target_type) - { - if (output_pins.empty() || std::find(output_pins.begin(), output_pins.end(), ep_pin) != output_pins.end()) - { - if (input_pins.empty() || std::find(input_pins.begin(), input_pins.end(), ep->get_pin()) != input_pins.end()) - { - if (!filter || filter(ep->get_gate())) - { - return true; - } - } - } - } - return false; - }); - - if (successors.size() > 1) - { - log_debug("netlist_utils", - "detected more than one valid successor gate for gate '{}' with ID {} in netlist with ID {}.", - current_gate->get_name(), - current_gate->get_id(), - current_gate->get_netlist()->get_id()); - break; - } - else if (!successors.empty()) - { - Gate* suc_gate = successors.at(0)->get_gate(); - - if (visited_gates.find(suc_gate) != visited_gates.end()) - { - log_debug("netlist_utils", "detected a loop at gate with ID {}.", suc_gate->get_id()); - break; - } - - gate_chain.push_back(suc_gate); - visited_gates.insert(suc_gate); - current_gate = suc_gate; - found_next_gate = true; - } - } while (found_next_gate); - // move backwards - current_gate = start_gate; - do - { - found_next_gate = false; - - // check all eligable predecessors of current gate - std::vector predecessors = current_gate->get_predecessors([input_pins, output_pins, target_type, filter](const GatePin* ep_pin, Endpoint* ep) { - if (ep->get_gate()->get_type() == target_type) - { - if (input_pins.empty() || std::find(input_pins.begin(), input_pins.end(), ep_pin) != input_pins.end()) - { - if (output_pins.empty() || std::find(output_pins.begin(), output_pins.end(), ep->get_pin()) != output_pins.end()) - { - if (!filter || filter(ep->get_gate())) - { - return true; - } - } - } - } - return false; - }); - - if (predecessors.size() > 1) - { - log_debug("netlist_utils", - "detected more than one valid predecessor gate for gate '{}' with ID {} in netlist with ID {}.", - current_gate->get_name(), - current_gate->get_id(), - current_gate->get_netlist()->get_id()); - break; - } - else if (!predecessors.empty()) - { - Gate* pred_gate = predecessors.at(0)->get_gate(); - - if (visited_gates.find(pred_gate) != visited_gates.end()) - { - log_debug("netlist_utils", "detected a loop at gate with ID {}.", pred_gate->get_id()); - break; - } - - gate_chain.push_front(pred_gate); - visited_gates.insert(pred_gate); - current_gate = pred_gate; - found_next_gate = true; - log_debug("netlist_utils", "found predecessor gate with ID {}.", pred_gate->get_id()); - } - } while (found_next_gate); - - return OK(std::vector(gate_chain.begin(), gate_chain.end())); - } - - Result> get_complex_gate_chain(Gate* start_gate, - const std::vector& chain_types, - const std::map>& input_pins, - const std::map>& output_pins, - const std::function& filter) - { - if (start_gate == nullptr) - { - return ERR("could not detect gate chain at start gate: start gate is a 'nullptr'"); - } - if (chain_types.size() < 2) - { - return ERR("could not detect gate chain at start gate: 'chain_types' comprises less than two target gate types"); - } - if (start_gate->get_type() != chain_types.at(0)) - { - return ERR("could not detect gate chain at start gate '" + start_gate->get_name() + "' with ID " + std::to_string(start_gate->get_id()) + ": start gate is not of type '" - + chain_types.front()->get_name() + "'"); - } - if (filter && !filter(start_gate)) - { - return ERR("could not detect gate chain at start gate '" + start_gate->get_name() + "' with ID " + std::to_string(start_gate->get_id()) - + ": filter evaluates to 'false' for start gate"); - } - - std::deque gate_chain = {start_gate}; - std::unordered_set visited_gates; - - u32 last_index = 0; - u32 current_index = (last_index + 1) % chain_types.size(); - - // move forward - bool found_next_gate; - const Gate* current_gate = start_gate; - do - { - found_next_gate = false; - - // check all successors of current gate - GateType* target_type = chain_types.at(current_index); - const std::vector& inputs = input_pins.at(target_type); - const std::vector& outputs = output_pins.at(chain_types.at(last_index)); - std::vector successors = current_gate->get_successors([target_type, inputs, outputs, filter](const GatePin* ep_pin, Endpoint* ep) { - if (ep->get_gate()->get_type() == target_type) - { - if (outputs.empty() || std::find(outputs.begin(), outputs.end(), ep_pin) != outputs.end()) - { - if (inputs.empty() || std::find(inputs.begin(), inputs.end(), ep->get_pin()) != inputs.end()) - { - if (!filter || filter(ep->get_gate())) - { - return true; - } - } - } - } - return false; - }); - - if (successors.size() > 1) - { - log_debug("netlist_utils", - "detected more than one valid successor gate for gate '{}' with ID {} in netlist with ID {}.", - current_gate->get_name(), - current_gate->get_id(), - current_gate->get_netlist()->get_id()); - break; - } - else if (!successors.empty()) - { - Gate* suc_gate = successors.at(0)->get_gate(); - - if (visited_gates.find(suc_gate) != visited_gates.end()) - { - log_debug("netlist_utils", "detected a loop at gate with ID {}.", suc_gate->get_id()); - break; - } - - gate_chain.push_back(suc_gate); - visited_gates.insert(suc_gate); - current_gate = suc_gate; - last_index = current_index; - current_index = (current_index + 1) % chain_types.size(); - found_next_gate = true; - } - } while (found_next_gate); - - // remove partial sequences at the end of the chain - while (current_index != 0) - { - gate_chain.pop_back(); - current_index--; - } - - current_gate = start_gate; - last_index = 0; - current_index = chain_types.size() - 1; - - // move backwards - do - { - found_next_gate = false; - - // check all predecessors of current gate - GateType* target_type = chain_types.at(current_index); - const std::vector& inputs = input_pins.at(chain_types.at(last_index)); - const std::vector& outputs = output_pins.at(target_type); - std::vector predecessors = current_gate->get_predecessors([target_type, inputs, outputs, filter](const GatePin* ep_pin, Endpoint* ep) { - if (ep->get_gate()->get_type() == target_type) - { - if (inputs.empty() || std::find(inputs.begin(), inputs.end(), ep_pin) != inputs.end()) - { - if (outputs.empty() || std::find(outputs.begin(), outputs.end(), ep->get_pin()) != outputs.end()) - { - if (!filter || filter(ep->get_gate())) - { - return true; - } - } - } - } - return false; - }); - - if (predecessors.size() > 1) - { - log_debug("netlist_utils", - "detected more than one valid predecessor gate for gate '{}' with ID {} in netlist with ID {}.", - current_gate->get_name(), - current_gate->get_id(), - current_gate->get_netlist()->get_id()); - break; - } - else if (!predecessors.empty()) - { - Gate* pred_gate = predecessors.at(0)->get_gate(); - - if (visited_gates.find(pred_gate) != visited_gates.end()) - { - log_debug("netlist_utils", "detected a loop at gate with ID {}.", pred_gate->get_id()); - break; - } - - gate_chain.push_front(pred_gate); - visited_gates.insert(pred_gate); - current_gate = pred_gate; - last_index = current_index; - current_index = (current_index == 0) ? chain_types.size() - 1 : current_index - 1; - found_next_gate = true; - } - } while (found_next_gate); - - // remove partial sequences at the beginning of the chain - while (last_index != 0) - { - gate_chain.pop_front(); - last_index--; - } - - return OK(std::vector(gate_chain.begin(), gate_chain.end())); - } } // namespace netlist_utils } // namespace hal diff --git a/src/plugin_system/plugin_manager.cpp b/src/plugin_system/plugin_manager.cpp index 7c7d642544fe..f006b9a64fc0 100644 --- a/src/plugin_system/plugin_manager.cpp +++ b/src/plugin_system/plugin_manager.cpp @@ -27,8 +27,43 @@ namespace hal { namespace { + /** + * A loaded plugin and the library it was loaded from. + * + * Exists so that the library can be kept open when the map is torn down at exit, which a + * std::tuple of the two gives no way to express. See the destructor. + */ + struct LoadedPlugin + { + std::unique_ptr instance; + std::unique_ptr library; + + /** + * Lets go of the library rather than unloading it. + * + * This only does anything while the process is exiting: unload() moves both pointers out + * before it erases an entry, so on that path there is nothing left here to destroy. + * + * Unloading at exit is not merely pointless but wrong. The netlist parser and writer + * registries hold a std::function per plugin that provides one, those registries live in + * libhal_netlist and are destroyed after this map is, and closing the libraries here left + * every one of those functions pointing into memory that had just been unmapped, which + * segfaulted on the way out. + */ + ~LoadedPlugin() + { + (void)library.release(); + } + + LoadedPlugin() = default; + LoadedPlugin(LoadedPlugin&&) = default; + LoadedPlugin& operator=(LoadedPlugin&&) = default; + LoadedPlugin(const LoadedPlugin&) = delete; + LoadedPlugin& operator=(const LoadedPlugin&) = delete; + }; + // stores library and factory identified by plugin name) - std::unordered_map, std::unique_ptr>> m_loaded_plugins; + std::unordered_map m_loaded_plugins; // stores special features offered by plugin std::unordered_map> m_plugin_features; @@ -315,7 +350,7 @@ namespace hal } m_current_loading.clear(); - m_loaded_plugins[plugin_name] = std::make_tuple(std::move(instance), std::move(lib)); + m_loaded_plugins[plugin_name] = LoadedPlugin{std::move(instance), std::move(lib)}; /* notify callback that a plugin was loaded*/ m_hook(true, plugin_name, file_path.string()); @@ -355,8 +390,8 @@ namespace hal log_info("core", "unloading plugin '{}'...", plugin_name); - auto rt_library = std::move(std::get<1>(loaded_it->second)); - auto plugin_inst = std::move(std::get<0>(loaded_it->second)); + auto rt_library = std::move(loaded_it->second.library); + auto plugin_inst = std::move(loaded_it->second.instance); { auto iplugType = dynamic_cast(plugin_inst.get()) ? 1 : 0; @@ -428,7 +463,7 @@ namespace hal return nullptr; } - auto instance = std::get<0>(it->second).get(); + auto instance = it->second.instance.get(); if (instance != nullptr && initialize) { instance->initialize(); @@ -440,7 +475,7 @@ namespace hal { for (const auto& [_, plugin] : m_loaded_plugins) { - if (auto* ui_plugin = dynamic_cast(std::get<0>(plugin).get()); ui_plugin != nullptr) + if (auto* ui_plugin = dynamic_cast(plugin.instance.get()); ui_plugin != nullptr) { return ui_plugin; } diff --git a/src/python_bindings/bindings/netlist_traversal_decorator.cpp b/src/python_bindings/bindings/netlist_traversal_decorator.cpp index a9efb8e6210e..4e68d020f429 100644 --- a/src/python_bindings/bindings/netlist_traversal_decorator.cpp +++ b/src/python_bindings/bindings/netlist_traversal_decorator.cpp @@ -13,6 +13,164 @@ namespace hal :param hal_py.Netlist netlist: The netlist to operate on. )"); + py::enum_(m, "TraversalDirection", R"(The direction in which a netlist is traversed.)") + .value("forward", TraversalDirection::forward, R"(Follow the fan-out, i.e., towards the successors of a gate.)") + .value("backward", TraversalDirection::backward, R"(Follow the fan-in, i.e., towards the predecessors of a gate.)") + .value("both", TraversalDirection::both, R"(Follow both directions.)") + .export_values(); + + py::enum_(m, "TraversalStop", R"(Where a traversal stops relative to the gates it is looking for.)") + .value("at_match", TraversalStop::at_match, R"(Stop at a gate the filter accepts, so the collected gates bound the search.)") + .value("at_mismatch", TraversalStop::at_mismatch, R"(Stop at a gate the filter rejects, so the collected gates form a connected region.)") + .value("never", TraversalStop::never, R"(Do not stop at a gate; bound the traversal with a depth or the endpoint filters.)") + .export_values(); + + py::class_ py_traversal_cache(m, "TraversalCache", R"( + A reusable store for the results of one specific traversal, handed to ``NetlistTraversalDecorator.get_gates``. + + The traversal a cache belongs to is sealed in when it is created and the cache can only ever be used for exactly that traversal, which is what makes reuse sound. Create one with ``NetlistTraversalDecorator.make_traversal_cache`` and drop it when the netlist is modified. + )"); + + py_netlist_traversal_decorator.def( + "get_common_inputs", + [](NetlistTraversalDecorator& self, const std::vector& gates, u32 threshold = 0) -> std::optional> { + auto res = self.get_common_inputs(gates, threshold); + if (res.is_ok()) + { + return res.get(); + } + log_error("python_context", "{}", res.get_error().get()); + return std::nullopt; + }, + py::arg("gates"), + py::arg("threshold") = 0, + borrowed(), + R"( + Get the nets that are inputs to at least ``threshold`` of the given gates. + + Shared inputs across a group of gates typically indicate a shared control signal, so this is a cheap way to test whether a set of gates belongs together. Nets driven by GND or VCC gates do not count. + + :param list[hal_py.Gate] gates: The gates to inspect. + :param int threshold: The number of gates a net has to feed. ``0`` to require all of them. Defaults to ``0``. + :returns: The common input nets on success, ``None`` otherwise. + :rtype: list[hal_py.Net] or None + )"); + + py_netlist_traversal_decorator.def( + "make_traversal_cache", + [](NetlistTraversalDecorator& self, + TraversalDirection direction, + const std::function& match, + TraversalStop stop, + const std::function& exit_endpoint_filter = nullptr, + const std::function& entry_endpoint_filter = nullptr) -> std::unique_ptr { + return std::make_unique(self.make_traversal_cache(direction, match, stop, exit_endpoint_filter, entry_endpoint_filter)); + }, + py::arg("direction"), + py::arg("match"), + py::arg("stop"), + py::arg("exit_endpoint_filter") = nullptr, + py::arg("entry_endpoint_filter") = nullptr, + R"( + Create a cache for one specific traversal, to be handed to ``get_gates`` in place of the traversal's parameters. + + The direction must be ``TraversalDirection.forward`` or ``backward``. The endpoint filters receive no depth and there is no depth limit, as either would make the cached answers depend on how a net was reached. + + :param hal_py.TraversalDirection direction: The direction to traverse in. + :param lambda match: The condition a gate has to meet to be collected. + :param hal_py.TraversalStop stop: Where to stop traversing, relative to the gates that ``match`` accepts. + :param lambda exit_endpoint_filter: Condition that has to hold to leave a gate. + :param lambda entry_endpoint_filter: Condition that has to hold to enter a gate. + :returns: The cache. + :rtype: hal_py.TraversalCache + )"); + + py_netlist_traversal_decorator.def( + "get_gates", + [](NetlistTraversalDecorator& self, const Gate* gate, TraversalCache& cache) -> std::optional> { + auto res = self.get_gates(gate, cache); + if (res.is_ok()) + { + return res.get(); + } + log_error("python_context", "{}", res.get_error().get()); + return std::nullopt; + }, + py::arg("gate"), + py::arg("cache"), + borrowed(), + R"( + Traverse the netlist from the given gate, sharing results through the cache: what an earlier call worked out is not walked again. + + :param hal_py.Gate gate: The gate to start from. + :param hal_py.TraversalCache cache: The cache holding the traversal and its results. + :returns: The collected gates on success, ``None`` otherwise. + :rtype: set[hal_py.Gate] or None + )"); + + py_netlist_traversal_decorator.def( + "get_gates", + [](NetlistTraversalDecorator& self, const Net* net, TraversalCache& cache) -> std::optional> { + auto res = self.get_gates(net, cache); + if (res.is_ok()) + { + return res.get(); + } + log_error("python_context", "{}", res.get_error().get()); + return std::nullopt; + }, + py::arg("net"), + py::arg("cache"), + borrowed(), + R"( + Traverse the netlist from the given net, sharing results through the cache: what an earlier call worked out is not walked again. + + :param hal_py.Net net: The net to start from. + :param hal_py.TraversalCache cache: The cache holding the traversal and its results. + :returns: The collected gates on success, ``None`` otherwise. + :rtype: set[hal_py.Gate] or None + )"); + + py_netlist_traversal_decorator.def( + "get_gates", + [](NetlistTraversalDecorator& self, + const Gate* gate, + TraversalDirection direction, + const std::function& match, + TraversalStop stop, + u32 max_depth = 0, + const std::function& exit_endpoint_filter = nullptr, + const std::function& entry_endpoint_filter = nullptr) -> std::optional> { + auto res = self.get_gates(gate, direction, match, stop, max_depth, exit_endpoint_filter, entry_endpoint_filter); + if (res.is_ok()) + { + return res.get(); + } + log_error("python_context", "{}", res.get_error().get()); + return std::nullopt; + }, + py::arg("gate"), + py::arg("direction"), + py::arg("match"), + py::arg("stop"), + py::arg("max_depth") = 0, + py::arg("exit_endpoint_filter") = nullptr, + py::arg("entry_endpoint_filter") = nullptr, + borrowed(), + R"( + Traverse the netlist from the given gate, collecting the gates that ``match`` accepts. + + :param hal_py.Gate gate: The gate to start from. + :param hal_py.TraversalDirection direction: The direction to traverse in. + :param lambda match: The condition a gate has to meet to be collected. + :param hal_py.TraversalStop stop: Where to stop traversing, relative to the gates that ``match`` accepts. + :param int max_depth: The maximum number of gates to traverse through. ``0`` for no limit. + :param lambda exit_endpoint_filter: Condition that has to hold to leave a gate. + :param lambda entry_endpoint_filter: Condition that has to hold to enter a gate. + :returns: The collected gates on success, ``None`` otherwise. + :rtype: set[hal_py.Gate] or None + )"); + py_netlist_traversal_decorator.def( "get_next_matching_gates", [](NetlistTraversalDecorator& self, @@ -254,7 +412,7 @@ namespace hal py_netlist_traversal_decorator.def( "get_next_sequential_gates", [](NetlistTraversalDecorator& self, const Net* net, bool successors, const std::set& forbidden_pins) -> std::optional> { - auto res = self.get_next_sequential_gates(net, successors, forbidden_pins, nullptr); + auto res = self.get_next_sequential_gates(net, successors, forbidden_pins); if (res.is_ok()) { return res.get(); @@ -267,7 +425,7 @@ namespace hal }, py::arg("net"), py::arg("successors"), - py::arg("forbidden_pins"), + py::arg("forbidden_pins") = std::set(), borrowed(), R"( Starting from the given net, traverse the netlist and return only the next layer of sequential successor/predecessor gates. Traverse over gates that are not sequential until a sequential gate is found. @@ -283,7 +441,7 @@ namespace hal py_netlist_traversal_decorator.def( "get_next_sequential_gates", [](NetlistTraversalDecorator& self, const Gate* gate, bool successors, const std::set& forbidden_pins) -> std::optional> { - auto res = self.get_next_sequential_gates(gate, successors, forbidden_pins, nullptr); + auto res = self.get_next_sequential_gates(gate, successors, forbidden_pins); if (res.is_ok()) { return res.get(); @@ -296,7 +454,7 @@ namespace hal }, py::arg("gate"), py::arg("successors"), - py::arg("forbidden_pins"), + py::arg("forbidden_pins") = std::set(), borrowed(), R"( Starting from the given gate, traverse the netlist and return only the next layer of sequential successor/predecessor gates. Traverse over gates that are not sequential until a sequential gate is found. @@ -337,62 +495,62 @@ namespace hal )"); py_netlist_traversal_decorator.def( - "get_next_combinational_gates", + "get_combinational_cone", [](NetlistTraversalDecorator& self, const Net* net, bool successors, const std::set& forbidden_pins) -> std::optional> { - auto res = self.get_next_combinational_gates(net, successors, forbidden_pins, nullptr); + auto res = self.get_combinational_cone(net, successors, forbidden_pins); if (res.is_ok()) { return res.get(); } else { - log_error("python_context", "error encountered while getting next combinational gates:\n{}", res.get_error().get()); + log_error("python_context", "error encountered while getting combinational cone:\n{}", res.get_error().get()); return std::nullopt; } }, py::arg("net"), py::arg("successors"), - py::arg("forbidden_pins"), + py::arg("forbidden_pins") = std::set(), borrowed(), R"( - Starting from the given net, traverse the netlist and return all combinational successor/predecessor gates. - Continue traversal as long as further combinational gates are found and stop at gates that are not combinational. + Starting from the given net, collect the combinational cone in the given direction, i.e., the combinational fan-out (``successors = True``) or fan-in (``successors = False``) of the net. + Continue traversal as long as further combinational gates are found and stop at gates that are not combinational, so that the cone extends up to (but not including) the sequential boundary. All combinational gates found during traversal are added to the result. Forbidden pins can be provided to, e.g., avoid the inclusion of logic in front of flip-flop control inputs. :param hal_py.Net net: Start net. - :param bool successors: Set ``True`` to get successors, set ``False`` to get predecessors. + :param bool successors: Set ``True`` to get the fan-out cone, set ``False`` to get the fan-in cone. :param set[hal_py.PinType] forbidden_pins: Traversal stops at pins of these types, i.e., gates reached through such a pin are not part of the result. Defaults to an empty set. - :returns: The next combinational gates on success, ``None`` otherwise. + :returns: The gates of the combinational cone on success, ``None`` otherwise. :rtype: set[hal_py.Gate] or None )"); py_netlist_traversal_decorator.def( - "get_next_combinational_gates", + "get_combinational_cone", [](NetlistTraversalDecorator& self, const Gate* gate, bool successors, const std::set& forbidden_pins) -> std::optional> { - auto res = self.get_next_combinational_gates(gate, successors, forbidden_pins, nullptr); + auto res = self.get_combinational_cone(gate, successors, forbidden_pins); if (res.is_ok()) { return res.get(); } else { - log_error("python_context", "error encountered while getting next combinational gates:\n{}", res.get_error().get()); + log_error("python_context", "error encountered while getting combinational cone:\n{}", res.get_error().get()); return std::nullopt; } }, py::arg("gate"), py::arg("successors"), - py::arg("forbidden_pins"), + py::arg("forbidden_pins") = std::set(), borrowed(), R"( - Starting from the given gate, traverse the netlist and return all combinational successor/predecessor gates. - Continue traversal as long as further combinational gates are found and stop at gates that are not combinational. + Starting from the given gate, collect the combinational cone in the given direction, i.e., the combinational fan-out (``successors = True``) or fan-in (``successors = False``) of the gate. + Continue traversal as long as further combinational gates are found and stop at gates that are not combinational, so that the cone extends up to (but not including) the sequential boundary. All combinational gates found during traversal are added to the result. Forbidden pins can be provided to, e.g., avoid the inclusion of logic in front of flip-flop control inputs. :param hal_py.Gate gate: Start gate. - :param bool successors: Set ``True`` to get successors, set ``False`` to get predecessors. + :param bool successors: Set ``True`` to get the fan-out cone, set ``False`` to get the fan-in cone. :param set[hal_py.PinType] forbidden_pins: Traversal stops at pins of these types, i.e., gates reached through such a pin are not part of the result. Defaults to an empty set. - :returns: The next combinational gates on success, ``None`` otherwise. + :returns: The gates of the combinational cone on success, ``None`` otherwise. :rtype: set[hal_py.Gate] or None )"); @@ -472,5 +630,148 @@ namespace hal :returns: A list of gates that connect the start with end gate on success, ``None`` otherwise. :rtype: list[hal_py.Gate] or None )"); + + py_netlist_traversal_decorator.def( + "get_shortest_path", + [](NetlistTraversalDecorator& self, + const Gate* start_gate, + const Module* end_module, + const PinDirection& direction, + const std::function& exit_endpoint_filter = nullptr, + const std::function& entry_endpoint_filter = nullptr) -> std::optional> { + auto res = self.get_shortest_path(start_gate, end_module, direction, exit_endpoint_filter, entry_endpoint_filter); + if (res.is_ok()) + { + return res.get(); + } + log_error("python_context", "{}", res.get_error().get()); + return std::nullopt; + }, + py::arg("start_gate"), + py::arg("end_module"), + py::arg("direction"), + py::arg("exit_endpoint_filter") = nullptr, + py::arg("entry_endpoint_filter") = nullptr, + borrowed(), + R"( + Find the shortest path that connects the start gate with any gate of the given module. + + :param hal_py.Gate start_gate: The gate to start from. + :param hal_py.Module end_module: The module to connect to. Gates of its submodules count as belonging to it. + :param hal_py.PinDirection direction: The direction to search in. + :param lambda exit_endpoint_filter: Filter condition that determines whether to stop traversal on a fan-in/out endpoint. + :param lambda entry_endpoint_filter: Filter condition that determines whether to stop traversal on a successor/predecessor endpoint. + :returns: The path on success, ``None`` otherwise. + :rtype: list[hal_py.Gate] or None + )"); + + py_netlist_traversal_decorator.def( + "get_shortest_path", + [](NetlistTraversalDecorator& self, + const Module* start_module, + const Module* end_module, + const PinDirection& direction, + const std::function& exit_endpoint_filter = nullptr, + const std::function& entry_endpoint_filter = nullptr) -> std::optional>> { + auto res = self.get_shortest_path(start_module, end_module, direction, exit_endpoint_filter, entry_endpoint_filter); + if (res.is_ok()) + { + return res.get(); + } + log_error("python_context", "{}", res.get_error().get()); + return std::nullopt; + }, + py::arg("start_module"), + py::arg("end_module"), + py::arg("direction"), + py::arg("exit_endpoint_filter") = nullptr, + py::arg("entry_endpoint_filter") = nullptr, + borrowed(), + R"( + Find every shortest path that connects the start module with the end module. + + :param hal_py.Module start_module: The module to start from. Gates of its submodules count as belonging to it. + :param hal_py.Module end_module: The module to connect to. Gates of its submodules count as belonging to it. + :param hal_py.PinDirection direction: The direction to search in. + :param lambda exit_endpoint_filter: Filter condition that determines whether to stop traversal on a fan-in/out endpoint. + :param lambda entry_endpoint_filter: Filter condition that determines whether to stop traversal on a successor/predecessor endpoint. + :returns: The shortest paths on success, ``None`` otherwise. + :rtype: list[list[hal_py.Gate]] or None + )"); + +py_netlist_traversal_decorator.def( + "get_gate_chain", + [](NetlistTraversalDecorator& self, Gate* start_gate, const std::vector& input_pins = {}, const std::vector& output_pins = {}, const std::function& filter = nullptr) + -> std::vector { + auto res = self.get_gate_chain(start_gate, input_pins, output_pins, filter); + if (res.is_ok()) + { + return res.get(); + } + else + { + log_error("python_context", "error encountered while detecting gate chain:\n{}", res.get_error().get()); + return {}; + } + }, + py::arg("start_gate"), + py::arg("input_pins") = std::vector(), + py::arg("output_pins") = std::vector(), + py::arg("filter") = nullptr, + borrowed(), + R"( + Find a sequence of identical gates that are connected via the specified input and output pins. + The start gate may be any gate within a such a sequence, it is not required to be the first or the last gate. + If input and/or output pins are specified, the gates must be connected through one of the input pins and/or one of the output pins. + The optional filter is evaluated on every gate such that the result only contains gates matching the specified condition. + + :param hal_py.Gate start_gate: The gate at which to start the chain detection. + :param list[hal_py.GatePin] input_pins: The input pins through which the gates must be connected. Defaults to an empty list. + :param set[hal_py.GatePin] output_pins: The output pins through which the gates must be connected. Defaults to an empty list. + :param lambda filter: An optional filter function to be evaluated on each gate. + :returns: A list of gates that form a chain on success, an empty list on error. + :rtype: list[hal_py.Gate] + )"); + + py_netlist_traversal_decorator.def( + "get_complex_gate_chain", + [](NetlistTraversalDecorator& self, Gate* start_gate, + const std::vector& chain_types, + const std::map>& input_pins, + const std::map>& output_pins, + const std::function& filter = nullptr) -> std::vector { + auto res = self.get_complex_gate_chain(start_gate, chain_types, input_pins, output_pins, filter); + if (res.is_ok()) + { + return res.get(); + } + else + { + log_error("python_context", "error encountered while detecting complex gate chain:\n{}", res.get_error().get()); + return {}; + } + }, + py::arg("start_gate"), + py::arg("chain_types"), + py::arg("input_pins"), + py::arg("output_pins"), + py::arg("filter") = nullptr, + borrowed(), + R"( + Find a sequence of gates (of the specified sequence of gate types) that are connected via the specified input and output pins. + The start gate may be any gate within a such a sequence, it is not required to be the first or the last gate. + However, the start gate must be of the first gate type within the repeating sequence. + If input and/or output pins are specified for a gate type, the gates must be connected through one of the input pins and/or one of the output pins. + The optional filter is evaluated on every gate such that the result only contains gates matching the specified condition. + + :param hal_py.Gate start_gate: The gate at which to start the chain detection. + :param list[hal_py.GateType] chain_types: The sequence of gate types that is expected to make up the gate chain. + :param dict[hal_py.GateType,set[str]] input_pins: The input pins (of every gate type of the sequence) through which the gates must be connected. + :param dict[hal_py.GateType,set[str]] output_pins: The output pins (of every gate type of the sequence) through which the gates must be connected. + :param lambda filter: An optional filter function to be evaluated on each gate. + :returns: A list of gates that form a chain on success, an empty list on error. + :rtype: list[hal_py.Gate] + )"); + } } // namespace hal \ No newline at end of file diff --git a/src/python_bindings/bindings/netlist_utils.cpp b/src/python_bindings/bindings/netlist_utils.cpp index 114d9564d3d6..3558cf99a1b7 100644 --- a/src/python_bindings/bindings/netlist_utils.cpp +++ b/src/python_bindings/bindings/netlist_utils.cpp @@ -2,6 +2,20 @@ namespace hal { + namespace + { + /// Tells a Python caller once per function and process that what they called is going away. + /// C++ callers get this from [[deprecated]] at compile time; a script has no compiler to warn it. + void warn_deprecated(const std::string& name, const std::string& use_instead) + { + static std::set warned; + if (warned.insert(name).second) + { + log_warning("python_context", "hal_py.NetlistUtils.{} is deprecated and will be removed in a future version, use {} instead.", name, use_instead); + } + } + } // namespace + void netlist_utils_init(py::module& m) { auto py_netlist_utils = m.def_submodule("NetlistUtils", R"( @@ -11,6 +25,7 @@ namespace hal py_netlist_utils.def( "get_subgraph_function", [](const Net* net, const std::vector& subgraph_gates) -> BooleanFunction { + warn_deprecated("get_subgraph_function", "hal_py.SubgraphNetlistDecorator.get_subgraph_function"); auto res = netlist_utils::get_subgraph_function(net, subgraph_gates); if (res.is_ok()) { @@ -35,7 +50,8 @@ namespace hal )"); py_netlist_utils.def( - "copy_netlist", [](const Netlist* nl) { return std::shared_ptr(netlist_utils::copy_netlist(nl)); }, py::arg("nl"), R"( + "copy_netlist", [](const Netlist* nl) { + warn_deprecated("copy_netlist", "hal_py.Netlist.copy"); return std::shared_ptr(netlist_utils::copy_netlist(nl)); }, py::arg("nl"), R"( Get a deep copy of an entire netlist including all of its gates, nets, modules, and groupings. :param hal_py.Netlist nl: The netlist to copy. @@ -43,7 +59,10 @@ namespace hal :rtype: hal_py.Netlist )"); - py_netlist_utils.def("get_ff_dependency_matrix", &netlist_utils::get_ff_dependency_matrix, py::arg("nl"), R"( + py_netlist_utils.def("get_ff_dependency_matrix", [](const Netlist* nl) { + warn_deprecated("get_ff_dependency_matrix", "boolean_influence.get_ff_dependency_matrix"); + return netlist_utils::get_ff_dependency_matrix(nl); + }, py::arg("nl"), borrowed(), R"( Get the FF dependency matrix of a netlist. :param hal_py.Netlist nl: The netlist to extract the dependency matrix from. @@ -51,12 +70,14 @@ namespace hal :rtype: tuple(dict[int,hal_py.Gate], list[list[int]]) )"); - py_netlist_utils.def("get_next_gates", - py::overload_cast&>(&netlist_utils::get_next_gates), + py_netlist_utils.def("get_next_gates", [](const Gate* a0, bool a1, int a2, const std::function& a3) { + warn_deprecated("get_next_gates", "hal_py.NetlistTraversalDecorator.get_next_matching_gates_until_depth"); + return netlist_utils::get_next_gates(a0, a1, a2, a3); + }, py::arg("gate"), py::arg("get_successors"), py::arg("depth") = 0, - py::arg("filter") = nullptr, + py::arg("filter") = nullptr, borrowed(), R"( Find predecessors or successors of a gate. If depth is set to 1 only direct predecessors/successors will be returned. Higher number of depth causes as many steps of recursive calls. @@ -73,12 +94,14 @@ namespace hal :rtype: list[hal_py.Gate] )"); - py_netlist_utils.def("get_next_gates", - py::overload_cast&>(&netlist_utils::get_next_gates), + py_netlist_utils.def("get_next_gates", [](const Net* a0, bool a1, int a2, const std::function& a3) { + warn_deprecated("get_next_gates", "hal_py.NetlistTraversalDecorator.get_next_matching_gates_until_depth"); + return netlist_utils::get_next_gates(a0, a1, a2, a3); + }, py::arg("net"), py::arg("get_successors"), py::arg("depth") = 0, - py::arg("filter") = nullptr, + py::arg("filter") = nullptr, borrowed(), R"( Find predecessors or successors of a net. If depth is set to 1 only direct predecessors/successors will be returned. Higher number of depth causes as many steps of recursive calls. @@ -94,11 +117,13 @@ namespace hal :rtype: list[hal_py.Gate] )"); - py_netlist_utils.def("get_next_sequential_gates", - py::overload_cast>&>(&netlist_utils::get_next_sequential_gates), + py_netlist_utils.def("get_next_sequential_gates", [](const Gate* a0, bool a1, std::unordered_map>& a2) { + warn_deprecated("get_next_sequential_gates", "hal_py.NetlistTraversalDecorator.get_next_sequential_gates"); + return netlist_utils::get_next_sequential_gates(a0, a1, a2); + }, py::arg("gate"), py::arg("get_successors"), - py::arg("cache"), + py::arg("cache"), borrowed(), R"( Find all sequential predecessors or successors of a gate. Traverses combinational logic of all input or output nets until sequential gates are found. @@ -114,7 +139,10 @@ namespace hal :rtype: list[hal_py.Gate] )"); - py_netlist_utils.def("get_next_sequential_gates", py::overload_cast(&netlist_utils::get_next_sequential_gates), py::arg("gate"), py::arg("get_successors"), R"( + py_netlist_utils.def("get_next_sequential_gates", [](const Gate* a0, bool a1) { + warn_deprecated("get_next_sequential_gates", "hal_py.NetlistTraversalDecorator.get_next_sequential_gates"); + return netlist_utils::get_next_sequential_gates(a0, a1); + }, py::arg("gate"), py::arg("get_successors"), borrowed(), R"( Find all sequential predecessors or successors of a gate. Traverses combinational logic of all input or output nets until sequential gates are found. The result may include the provided gate itself. @@ -125,11 +153,13 @@ namespace hal :rtype: list[hal_py.Gate] )"); - py_netlist_utils.def("get_next_sequential_gates", - py::overload_cast>&>(&netlist_utils::get_next_sequential_gates), + py_netlist_utils.def("get_next_sequential_gates", [](const Net* a0, bool a1, std::unordered_map>& a2) { + warn_deprecated("get_next_sequential_gates", "hal_py.NetlistTraversalDecorator.get_next_sequential_gates"); + return netlist_utils::get_next_sequential_gates(a0, a1, a2); + }, py::arg("net"), py::arg("get_successors"), - py::arg("cache"), + py::arg("cache"), borrowed(), R"( Find all sequential predecessors or successors of a net. Traverses combinational logic of all input or output nets until sequential gates are found. @@ -144,7 +174,10 @@ namespace hal :rtype: list[hal_py.Net] )"); - py_netlist_utils.def("get_next_sequential_gates", py::overload_cast(&netlist_utils::get_next_sequential_gates), py::arg("net"), py::arg("get_successors"), R"( + py_netlist_utils.def("get_next_sequential_gates", [](const Net* a0, bool a1) { + warn_deprecated("get_next_sequential_gates", "hal_py.NetlistTraversalDecorator.get_next_sequential_gates"); + return netlist_utils::get_next_sequential_gates(a0, a1); + }, py::arg("net"), py::arg("get_successors"), borrowed(), R"( Find all sequential predecessors or successors of a net. Traverses combinational logic of all input or output nets until sequential gates are found. @@ -154,12 +187,14 @@ namespace hal :rtype: list[hal_py.Net] )"); - py_netlist_utils.def("get_path", - py::overload_cast, std::unordered_map>&>(&netlist_utils::get_path), + py_netlist_utils.def("get_path", [](const Gate* a0, bool a1, std::set a2, std::unordered_map>& a3) { + warn_deprecated("get_path", "hal_py.NetlistTraversalDecorator.get_gates with the negated condition and TraversalStop.at_mismatch"); + return netlist_utils::get_path(a0, a1, a2, a3); + }, py::arg("gate"), py::arg("get_successors"), py::arg("stop_properties"), - py::arg("cache"), + py::arg("cache"), borrowed(), R"( Find all gates on the predecessor or successor path of a gate. Traverses all input or output nets until gates of the specified base types are found. @@ -177,7 +212,10 @@ namespace hal )"); py_netlist_utils.def( - "get_path", py::overload_cast>(&netlist_utils::get_path), py::arg("gate"), py::arg("get_successors"), py::arg("stop_properties"), R"( + "get_path", [](const Gate* a0, bool a1, std::set a2) { + warn_deprecated("get_path", "hal_py.NetlistTraversalDecorator.get_gates with the negated condition and TraversalStop.at_mismatch"); + return netlist_utils::get_path(a0, a1, a2); + }, py::arg("gate"), py::arg("get_successors"), py::arg("stop_properties"), borrowed(), R"( Find all gates on the predeccessor or successor path of a gate. Traverses all input or output nets until gates of the specified base types are found. The result may include the provided gate itself. @@ -189,12 +227,14 @@ namespace hal :rtype: list[hal_py.Gate] )"); - py_netlist_utils.def("get_path", - py::overload_cast, std::unordered_map>&>(&netlist_utils::get_path), + py_netlist_utils.def("get_path", [](const Net* a0, bool a1, std::set a2, std::unordered_map>& a3) { + warn_deprecated("get_path", "hal_py.NetlistTraversalDecorator.get_gates with the negated condition and TraversalStop.at_mismatch"); + return netlist_utils::get_path(a0, a1, a2, a3); + }, py::arg("net"), py::arg("get_successors"), py::arg("stop_properties"), - py::arg("cache"), + py::arg("cache"), borrowed(), R"( Find all gates on the predecessor or successor path of a net. Traverses all input or output nets until gates of the specified base types are found. @@ -210,7 +250,10 @@ namespace hal :rtype: list[hal_py.Net] )"); py_netlist_utils.def( - "get_path", py::overload_cast>(&netlist_utils::get_path), py::arg("net"), py::arg("get_successors"), py::arg("stop_properties"), R"( + "get_path", [](const Net* a0, bool a1, std::set a2) { + warn_deprecated("get_path", "hal_py.NetlistTraversalDecorator.get_gates with the negated condition and TraversalStop.at_mismatch"); + return netlist_utils::get_path(a0, a1, a2); + }, py::arg("net"), py::arg("get_successors"), py::arg("stop_properties"), borrowed(), R"( Find all gates on the predecessor or successor path of a net. Traverses all input or output nets until gates of the specified base types are found. @@ -221,7 +264,10 @@ namespace hal :rtype: list[hal_py.Net] )"); - py_netlist_utils.def("get_nets_at_pins", netlist_utils::get_nets_at_pins, py::arg("gate"), py::arg("pins"), R"( + py_netlist_utils.def("get_nets_at_pins", [](Gate* gate, std::vector pins) { + warn_deprecated("get_nets_at_pins", "hal_py.Gate.get_fan_in_net or get_fan_out_net per pin"); + return netlist_utils::get_nets_at_pins(gate, pins); + }, py::arg("gate"), py::arg("pins"), borrowed(), R"( Get the nets that are connected to a subset of pins of the specified gate. :param hal_py.Gate gate: The gate. @@ -233,6 +279,7 @@ namespace hal py_netlist_utils.def( "remove_buffers", [](Netlist* netlist, bool analyze_inputs = false) -> i32 { + warn_deprecated("remove_buffers", "netlist_preprocessing.remove_buffers"); auto res = netlist_utils::remove_buffers(netlist, analyze_inputs); if (res.is_ok()) { @@ -259,6 +306,7 @@ namespace hal py_netlist_utils.def( "remove_unused_lut_endpoints", [](Netlist* netlist) -> i32 { + warn_deprecated("remove_unused_lut_endpoints", "netlist_preprocessing.remove_unused_lut_inputs"); auto res = netlist_utils::remove_unused_lut_endpoints(netlist); if (res.is_ok()) { @@ -279,7 +327,10 @@ namespace hal :rtype: int )"); - py_netlist_utils.def("get_common_inputs", &netlist_utils::get_common_inputs, py::arg("gates"), py::arg("threshold") = 0, R"( + py_netlist_utils.def("get_common_inputs", [](const std::vector& gates, u32 threshold) { + warn_deprecated("get_common_inputs", "hal_py.NetlistTraversalDecorator.get_common_inputs"); + return netlist_utils::get_common_inputs(gates, threshold); + }, py::arg("gates"), py::arg("threshold") = 0, borrowed(), R"( Returns all nets that are considered to be common inputs to the provided gates. A threshold value can be provided to specify the number of gates a net must be connected to in order to be classified as a common input. If the theshold value is set to 0, a net must be input to all gates to be considered a common input. @@ -293,6 +344,7 @@ namespace hal py_netlist_utils.def( "replace_gate", [](Gate* gate, GateType* target_type, std::map pin_map) -> i32 { + warn_deprecated("replace_gate", "hal_py.NetlistModificationDecorator.replace_gate"); auto res = netlist_utils::replace_gate(gate, target_type, pin_map); if (res.is_ok()) { @@ -319,79 +371,14 @@ namespace hal :rtype: bool )"); - py_netlist_utils.def( - "get_gate_chain", - [](Gate* start_gate, const std::vector& input_pins = {}, const std::vector& output_pins = {}, const std::function& filter = nullptr) - -> std::vector { - auto res = netlist_utils::get_gate_chain(start_gate, input_pins, output_pins, filter); - if (res.is_ok()) - { - return res.get(); - } - else - { - log_error("python_context", "error encountered while detecting gate chain:\n{}", res.get_error().get()); - return {}; - } - }, - py::arg("start_gate"), - py::arg("input_pins") = std::vector(), - py::arg("output_pins") = std::vector(), - py::arg("filter") = nullptr, - R"( - Find a sequence of identical gates that are connected via the specified input and output pins. - The start gate may be any gate within a such a sequence, it is not required to be the first or the last gate. - If input and/or output pins are specified, the gates must be connected through one of the input pins and/or one of the output pins. - The optional filter is evaluated on every gate such that the result only contains gates matching the specified condition. - - :param hal_py.Gate start_gate: The gate at which to start the chain detection. - :param list[hal_py.GatePin] input_pins: The input pins through which the gates must be connected. Defaults to an empty list. - :param set[hal_py.GatePin] output_pins: The output pins through which the gates must be connected. Defaults to an empty list. - :param lambda filter: An optional filter function to be evaluated on each gate. - :returns: A list of gates that form a chain on success, an empty list on error. - :rtype: list[hal_py.Gate] - )"); - py_netlist_utils.def( - "get_complex_gate_chain", - [](Gate* start_gate, - const std::vector& chain_types, - const std::map>& input_pins, - const std::map>& output_pins, - const std::function& filter = nullptr) -> std::vector { - auto res = netlist_utils::get_complex_gate_chain(start_gate, chain_types, input_pins, output_pins, filter); - if (res.is_ok()) - { - return res.get(); - } - else - { - log_error("python_context", "error encountered while detecting complex gate chain:\n{}", res.get_error().get()); - return {}; - } - }, - py::arg("start_gate"), - py::arg("chain_types"), - py::arg("input_pins"), - py::arg("output_pins"), - py::arg("filter") = nullptr, - R"( - Find a sequence of gates (of the specified sequence of gate types) that are connected via the specified input and output pins. - The start gate may be any gate within a such a sequence, it is not required to be the first or the last gate. - However, the start gate must be of the first gate type within the repeating sequence. - If input and/or output pins are specified for a gate type, the gates must be connected through one of the input pins and/or one of the output pins. - The optional filter is evaluated on every gate such that the result only contains gates matching the specified condition. - - :param hal_py.Gate start_gate: The gate at which to start the chain detection. - :param list[hal_py.GateType] chain_types: The sequence of gate types that is expected to make up the gate chain. - :param dict[hal_py.GateType,set[str]] input_pins: The input pins (of every gate type of the sequence) through which the gates must be connected. - :param dict[hal_py.GateType,set[str]] output_pins: The output pins (of every gate type of the sequence) through which the gates must be connected. - :param lambda filter: An optional filter function to be evaluated on each gate. - :returns: A list of gates that form a chain on success, an empty list on error. - :rtype: list[hal_py.Gate] - )"); - py_netlist_utils.def("get_shortest_path", py::overload_cast(&netlist_utils::get_shortest_path), py::arg("start_gate"), py::arg("end_gate"), py::arg("search_both_directions") = false, R"( + + + py_netlist_utils.def("get_shortest_path", [](Gate* a0, Gate* a1, bool a2) { + warn_deprecated("get_shortest_path", "hal_py.NetlistTraversalDecorator.get_shortest_path"); + return netlist_utils::get_shortest_path(a0, a1, a2); + }, py::arg("start_gate"), py::arg("end_gate"), py::arg("search_both_directions") = false, borrowed(), R"( Find the shortest path (i.e., the result set with the lowest number of gates) that connects the start gate with the end gate. The gate where the search started from will be the first in the result vector, the end gate will be the last. If there is no such path an empty vector is returned. If there is more than one path with the same length only the first one is returned. @@ -403,7 +390,10 @@ namespace hal :rtype: list[hal_py.Gate] )"); - py_netlist_utils.def("get_shortest_path", py::overload_cast(&netlist_utils::get_shortest_path), py::arg("start_gate"), py::arg("end_module"), py::arg("forward_direction"), R"( + py_netlist_utils.def("get_shortest_path", [](Gate* a0, Module* a1, bool a2) { + warn_deprecated("get_shortest_path", "hal_py.NetlistTraversalDecorator.get_shortest_path"); + return netlist_utils::get_shortest_path(a0, a1, a2); + }, py::arg("start_gate"), py::arg("end_module"), py::arg("forward_direction"), borrowed(), R"( Find the shortest path (i.e., the result set with the lowest number of gates) that connects the start gate with any gate from the given module. The gate where the search started from will be the first in the result vector, the end gate will be the last. If there is no such path an empty vector is returned. If there is more than one path with the same length only the first one is returned. @@ -415,7 +405,10 @@ namespace hal :rtype: list[hal_py.Gate] )"); - py_netlist_utils.def("get_shortest_path", py::overload_cast(&netlist_utils::get_shortest_path), py::arg("start_module"), py::arg("end_module"), R"( + py_netlist_utils.def("get_shortest_path", [](Module* a0, Module* a1) { + warn_deprecated("get_shortest_path", "hal_py.NetlistTraversalDecorator.get_shortest_path"); + return netlist_utils::get_shortest_path(a0, a1); + }, py::arg("start_module"), py::arg("end_module"), borrowed(), R"( Find the shortest path (i.e., the result set with the lowest number of gates) that connects the start module with the target module. There might be more than one connection thus a list of connecting gate lists is returned. diff --git a/src/python_bindings/bindings/smt.cpp b/src/python_bindings/bindings/smt.cpp index ec1d3ca83af2..136ebc227d7f 100644 --- a/src/python_bindings/bindings/smt.cpp +++ b/src/python_bindings/bindings/smt.cpp @@ -101,6 +101,20 @@ namespace hal :rtype: hal_py.SMT.QueryConfig )"); + py_smt_query_config.def("to_string", &SMT::QueryConfig::to_string, R"( + Translates the SMT query configuration into its string representation. + + :returns: A string representing the SMT query configuration. + :rtype: str + )"); + + py_smt_query_config.def("__str__", [](const SMT::QueryConfig& config) { return config.to_string(); }, R"( + Translates the SMT query configuration into its string representation. + + :returns: A string representing the SMT query configuration. + :rtype: str + )"); + py::class_ py_smt_constraint(py_smt, "Constraint", R"( Represents a constraint to the SMT query. A constraint is either an assignment of two Boolean functions or a single Boolean function, e.g., an equality check or similar. @@ -174,6 +188,20 @@ namespace hal :rtype: hal_py.BooleanFunction or None )"); + py_smt_constraint.def("to_string", &SMT::Constraint::to_string, R"( + Translates the SMT constraint into its string representation. + + :returns: A string representing the SMT constraint. + :rtype: str + )"); + + py_smt_constraint.def("__str__", [](const SMT::Constraint& constraint) { return constraint.to_string(); }, R"( + Translates the SMT constraint into its string representation. + + :returns: A string representing the SMT constraint. + :rtype: str + )"); + py::enum_ py_smt_result_type(py_smt, "SolverResultType", R"( Result type of an SMT solver query. )"); @@ -261,6 +289,20 @@ namespace hal :rtype: hal_py.BooleanFunction or None )"); + py_smt_model.def("to_string", &SMT::Model::to_string, R"( + Translates the SMT model into its string representation. + + :returns: A string representing the SMT model. + :rtype: str + )"); + + py_smt_model.def("__str__", [](const SMT::Model& model) { return model.to_string(); }, R"( + Translates the SMT model into its string representation. + + :returns: A string representing the SMT model. + :rtype: str + )"); + py::class_ py_smt_result(py_smt, "SolverResult", R"( Represents the result of an SMT query. )"); @@ -328,6 +370,20 @@ namespace hal :rtype: bool )"); + py_smt_result.def("to_string", &SMT::SolverResult::to_string, R"( + Translates the SMT result into its string representation. + + :returns: A string representing the SMT result. + :rtype: str + )"); + + py_smt_result.def("__str__", [](const SMT::SolverResult& result) { return result.to_string(); }, R"( + Translates the SMT result into its string representation. + + :returns: A string representing the SMT result. + :rtype: str + )"); + py::class_ py_smt_solver(py_smt, "Solver", R"( Provides an interface to query SMT solvers for a list of constraints, i.e. statements that have to be equal. To this end, we translate constraints to a SMT-LIB v2 string representation and query solvers with a defined configuration, i.e., chosen solver, model generation etc. )"); diff --git a/tests/netlist/decorators.cpp b/tests/netlist/decorators.cpp index 068788cec66a..49e5ff18d727 100644 --- a/tests/netlist/decorators.cpp +++ b/tests/netlist/decorators.cpp @@ -791,114 +791,109 @@ namespace hal { // successors { - const auto res = trav_dec.get_next_sequential_gates(dff0, true, {}, nullptr); + const auto res = trav_dec.get_next_sequential_gates(dff0, true, {}); EXPECT_TRUE(res.is_ok()); EXPECT_EQ(res.get(), std::set({dff0, dff4, dff5})); } { - const auto res = trav_dec.get_next_sequential_gates(dff2, true, {}, nullptr); + const auto res = trav_dec.get_next_sequential_gates(dff2, true, {}); EXPECT_TRUE(res.is_ok()); EXPECT_EQ(res.get(), std::set({dff5, dff6, dff7, dff3})); } { - const auto res = trav_dec.get_next_sequential_gates(dff4, true, {}, nullptr); + const auto res = trav_dec.get_next_sequential_gates(dff4, true, {}); EXPECT_TRUE(res.is_ok()); EXPECT_EQ(res.get(), std::set({dff8})); } { - const auto res = trav_dec.get_next_sequential_gates(dff8, true, {}, nullptr); + const auto res = trav_dec.get_next_sequential_gates(dff8, true, {}); EXPECT_TRUE(res.is_ok()); EXPECT_EQ(res.get(), std::set()); } { - const auto res = trav_dec.get_next_sequential_gates(sff0, true, {}, nullptr); + const auto res = trav_dec.get_next_sequential_gates(sff0, true, {}); EXPECT_TRUE(res.is_ok()); EXPECT_EQ(res.get(), std::set({sff1, dff0, dff1, dff2, dff3, dff4, dff5, dff6, dff7, dff8, dff9, dff10, dff11})); } { - const auto res = trav_dec.get_next_sequential_gates(sff0, true, {PinType::enable, PinType::reset}, nullptr); + const auto res = trav_dec.get_next_sequential_gates(sff0, true, {PinType::enable, PinType::reset}); EXPECT_TRUE(res.is_ok()); EXPECT_EQ(res.get(), std::set({sff1})); } { - const auto res = trav_dec.get_next_sequential_gates(sff1, true, {}, nullptr); + const auto res = trav_dec.get_next_sequential_gates(sff1, true, {}); EXPECT_TRUE(res.is_ok()); EXPECT_EQ(res.get(), std::set({dff0, dff1, dff2, dff3, dff4, dff5, dff6, dff7, dff8, dff9, dff10, dff11})); } { - const auto res = trav_dec.get_next_sequential_gates(sff1, true, {PinType::enable, PinType::reset}, nullptr); + const auto res = trav_dec.get_next_sequential_gates(sff1, true, {PinType::enable, PinType::reset}); EXPECT_TRUE(res.is_ok()); EXPECT_EQ(res.get(), std::set({})); } { - std::unordered_map> cache; - const auto res1 = trav_dec.get_next_sequential_gates(dff1, true, {}, &cache); + const auto res1 = trav_dec.get_next_sequential_gates(dff1, true, {}); EXPECT_TRUE(res1.is_ok()); EXPECT_EQ(res1.get(), std::set({dff4, dff5, dff6, dff0})); - const auto res2 = trav_dec.get_next_sequential_gates(dff2, true, {}, &cache); + const auto res2 = trav_dec.get_next_sequential_gates(dff2, true, {}); EXPECT_TRUE(res2.is_ok()); EXPECT_EQ(res2.get(), std::set({dff5, dff6, dff7, dff3})); } { - std::unordered_map> cache; - const auto res1 = trav_dec.get_next_sequential_gates(dff3, true, {}, &cache); + const auto res1 = trav_dec.get_next_sequential_gates(dff3, true, {}); EXPECT_TRUE(res1.is_ok()); EXPECT_EQ(res1.get(), std::set({dff6, dff7, dff3})); - const auto res2 = trav_dec.get_next_sequential_gates(dff2, true, {}, &cache); + const auto res2 = trav_dec.get_next_sequential_gates(dff2, true, {}); EXPECT_TRUE(res2.is_ok()); EXPECT_EQ(res2.get(), std::set({dff5, dff6, dff7, dff3})); } // predecessors { - const auto res = trav_dec.get_next_sequential_gates(dff4, false, {}, nullptr); + const auto res = trav_dec.get_next_sequential_gates(dff4, false, {}); EXPECT_TRUE(res.is_ok()); EXPECT_EQ(res.get(), std::set({dff0, dff1, sff0, sff1})); } { - const auto res = trav_dec.get_next_sequential_gates(dff4, false, {PinType::enable, PinType::reset, PinType::clock}, nullptr); + const auto res = trav_dec.get_next_sequential_gates(dff4, false, {PinType::enable, PinType::reset, PinType::clock}); EXPECT_TRUE(res.is_ok()); EXPECT_EQ(res.get(), std::set({dff0, dff1})); } { - const auto res = trav_dec.get_next_sequential_gates(dff5, false, {PinType::enable, PinType::reset, PinType::clock}, nullptr); + const auto res = trav_dec.get_next_sequential_gates(dff5, false, {PinType::enable, PinType::reset, PinType::clock}); EXPECT_TRUE(res.is_ok()); EXPECT_EQ(res.get(), std::set({dff0, dff1, dff2})); } { - const auto res = trav_dec.get_next_sequential_gates(dff0, false, {PinType::enable, PinType::reset, PinType::clock}, nullptr); + const auto res = trav_dec.get_next_sequential_gates(dff0, false, {PinType::enable, PinType::reset, PinType::clock}); EXPECT_TRUE(res.is_ok()); EXPECT_EQ(res.get(), std::set({dff0, dff1})); } { - std::unordered_map> cache; - const auto res1 = trav_dec.get_next_sequential_gates(dff5, false, {}, &cache); + const auto res1 = trav_dec.get_next_sequential_gates(dff5, false, {}); EXPECT_TRUE(res1.is_ok()); EXPECT_EQ(res1.get(), std::set({dff0, dff1, dff2, sff0, sff1})); - const auto res2 = trav_dec.get_next_sequential_gates(dff6, false, {}, &cache); + const auto res2 = trav_dec.get_next_sequential_gates(dff6, false, {}); EXPECT_TRUE(res2.is_ok()); EXPECT_EQ(res2.get(), std::set({dff1, dff2, dff3, sff0, sff1})); } { - std::unordered_map> cache; - const auto res1 = trav_dec.get_next_sequential_gates(dff5, false, {PinType::enable, PinType::reset, PinType::clock}, &cache); + const auto res1 = trav_dec.get_next_sequential_gates(dff5, false, {PinType::enable, PinType::reset, PinType::clock}); EXPECT_TRUE(res1.is_ok()); EXPECT_EQ(res1.get(), std::set({dff0, dff1, dff2})); - const auto res2 = trav_dec.get_next_sequential_gates(dff6, false, {PinType::enable, PinType::reset, PinType::clock}, &cache); + const auto res2 = trav_dec.get_next_sequential_gates(dff6, false, {PinType::enable, PinType::reset, PinType::clock}); EXPECT_TRUE(res2.is_ok()); EXPECT_EQ(res2.get(), std::set({dff1, dff2, dff3})); } { - std::unordered_map> cache; - const auto res1 = trav_dec.get_next_sequential_gates(dff6, false, {PinType::enable, PinType::reset, PinType::clock}, &cache); + const auto res1 = trav_dec.get_next_sequential_gates(dff6, false, {PinType::enable, PinType::reset, PinType::clock}); EXPECT_TRUE(res1.is_ok()); EXPECT_EQ(res1.get(), std::set({dff1, dff2, dff3})); - const auto res2 = trav_dec.get_next_sequential_gates(dff7, false, {PinType::enable, PinType::reset, PinType::clock}, &cache); + const auto res2 = trav_dec.get_next_sequential_gates(dff7, false, {PinType::enable, PinType::reset, PinType::clock}); EXPECT_TRUE(res2.is_ok()); EXPECT_EQ(res2.get(), std::set({dff2, dff3})); } @@ -996,59 +991,57 @@ namespace hal { } } { - // test NetlistModificationDecorator::get_next_combinational_gates + // test NetlistModificationDecorator::get_combinational_cone const auto trav_dec = NetlistTraversalDecorator(*(nl.get())); // successors { - const auto res = trav_dec.get_next_combinational_gates(dff4, true, {}, nullptr); + const auto res = trav_dec.get_combinational_cone(dff4, true, {}); EXPECT_TRUE(res.is_ok()); EXPECT_EQ(res.get(), std::set({inv2})); } { - const auto res = trav_dec.get_next_combinational_gates(dff0, true, {}, nullptr); + const auto res = trav_dec.get_combinational_cone(dff0, true, {}); EXPECT_TRUE(res.is_ok()); EXPECT_EQ(res.get(), std::set({inv0, and0, or2, or3, or0})); } { - std::unordered_map> cache; - const auto res1 = trav_dec.get_next_combinational_gates(dff1, true, {}, &cache); + const auto res1 = trav_dec.get_combinational_cone(dff1, true, {}); EXPECT_TRUE(res1.is_ok()); EXPECT_EQ(res1.get(), std::set({and0, or2, or3, and1, or4, or0})); - const auto res2 = trav_dec.get_next_combinational_gates(dff2, true, {}, &cache); + const auto res2 = trav_dec.get_combinational_cone(dff2, true, {}); EXPECT_TRUE(res2.is_ok()); EXPECT_EQ(res2.get(), std::set({and1, or3, or4, and2, or5, or1})); } // predecessors { - const auto res = trav_dec.get_next_combinational_gates(dff4, false, {}, nullptr); + const auto res = trav_dec.get_combinational_cone(dff4, false, {}); EXPECT_TRUE(res.is_ok()); EXPECT_EQ(res.get(), std::set({inv0, and0, or2, inv6})); } { - const auto res = trav_dec.get_next_combinational_gates(dff4, false, {PinType::enable, PinType::reset, PinType::clock, PinType::set}, nullptr); + const auto res = trav_dec.get_combinational_cone(dff4, false, {PinType::enable, PinType::reset, PinType::clock, PinType::set}); EXPECT_TRUE(res.is_ok()); EXPECT_EQ(res.get(), std::set({inv0, and0, or2})); } { - const auto res = trav_dec.get_next_combinational_gates(dff4, false, {PinType::none}, nullptr); + const auto res = trav_dec.get_combinational_cone(dff4, false, {PinType::none}); EXPECT_TRUE(res.is_ok()); EXPECT_EQ(res.get(), std::set({})); } { - const auto res = trav_dec.get_next_combinational_gates(dff0, false, {}, nullptr); + const auto res = trav_dec.get_combinational_cone(dff0, false, {}); EXPECT_TRUE(res.is_ok()); EXPECT_EQ(res.get(), std::set({inv0, and0, or2, or0, inv6})); } { - std::unordered_map> cache; - const auto res1 = trav_dec.get_next_combinational_gates(dff5, false, {}, &cache); + const auto res1 = trav_dec.get_combinational_cone(dff5, false, {}); EXPECT_TRUE(res1.is_ok()); EXPECT_EQ(res1.get(), std::set({or3, and0, and1, inv6})); - const auto res2 = trav_dec.get_next_combinational_gates(dff6, false, {}, &cache); + const auto res2 = trav_dec.get_combinational_cone(dff6, false, {}); EXPECT_TRUE(res2.is_ok()); EXPECT_EQ(res2.get(), std::set({or4, and1, and2, inv6})); } @@ -1056,4 +1049,470 @@ namespace hal { } TEST_END } + + /** + * Test the shortest path searches of the traversal decorator. + * + * These had no test at all, which is why they are written before the decorator is restructured: + * they describe what the searches do today so that a rewrite can be shown not to change it. + * + * Functions: get_shortest_path, get_shortest_path_distance + */ + TEST_F(DecoratorTest, check_netlist_traversal_decorator_shortest_path) + { + TEST_START + { + std::unique_ptr nl = test_utils::create_empty_netlist(); + ASSERT_NE(nl, nullptr); + Netlist* nl_raw = nl.get(); + const GateLibrary* gl = nl_raw->get_gate_library(); + + // A long way round and a short one between the same two gates: + // start -> a -> b -> finish (four gates) + // start -> c -> finish (three gates) + Gate* start = nl_raw->create_gate(gl->get_gate_type_by_name("BUF"), "start"); + Gate* a = nl_raw->create_gate(gl->get_gate_type_by_name("BUF"), "a"); + Gate* b = nl_raw->create_gate(gl->get_gate_type_by_name("BUF"), "b"); + Gate* c = nl_raw->create_gate(gl->get_gate_type_by_name("BUF"), "c"); + Gate* finish = nl_raw->create_gate(gl->get_gate_type_by_name("OR2"), "finish"); + + // one net out of start feeding both branches, as an output pin drives a single net + Net* fan = nl_raw->create_net("fan"); + fan->add_source(start, "O"); + fan->add_destination(a, "I"); + fan->add_destination(c, "I"); + + test_utils::connect(nl_raw, a, "O", b, "I", "a_to_b"); + test_utils::connect(nl_raw, b, "O", finish, "I0", "b_to_finish"); + test_utils::connect(nl_raw, c, "O", finish, "I1", "c_to_finish"); + + NetlistTraversalDecorator dec(*nl_raw); + + { + // the short branch wins, and the path runs from start to finish + auto res = dec.get_shortest_path(start, finish, PinDirection::output); + ASSERT_TRUE(res.is_ok()); + ASSERT_TRUE(res.get().has_value()); + const auto path = res.get().value(); + ASSERT_EQ(path.size(), 3); + EXPECT_EQ(path.front(), start); + EXPECT_EQ(path.at(1), c); + EXPECT_EQ(path.back(), finish); + } + { + auto res = dec.get_shortest_path_distance(start, finish, PinDirection::output); + ASSERT_TRUE(res.is_ok()); + ASSERT_TRUE(res.get().has_value()); + EXPECT_EQ(res.get().value(), 2); + } + { + // nothing leads backwards from start to finish + auto res = dec.get_shortest_path(start, finish, PinDirection::input); + ASSERT_TRUE(res.is_ok()); + EXPECT_FALSE(res.get().has_value()); + } + { + // searching both ways finds it regardless of which end it is asked from + auto res = dec.get_shortest_path(finish, start, PinDirection::inout); + ASSERT_TRUE(res.is_ok()); + ASSERT_TRUE(res.get().has_value()); + EXPECT_EQ(res.get().value().size(), 3); + } + { + // an endpoint filter that rejects everything leaves nothing to find + auto res = dec.get_shortest_path(start, finish, PinDirection::output, [](const Endpoint*, u32) { return false; }); + ASSERT_TRUE(res.is_ok()); + EXPECT_FALSE(res.get().has_value()); + } + { + // a gate that is not connected at all + Gate* island = nl_raw->create_gate(gl->get_gate_type_by_name("BUF"), "island"); + auto res = dec.get_shortest_path(start, island, PinDirection::output); + ASSERT_TRUE(res.is_ok()); + EXPECT_FALSE(res.get().has_value()); + } + } + TEST_END + } + + /** + * Test the shortest path searches that end at a module rather than at a gate. + * + * Functions: get_shortest_path + */ + TEST_F(DecoratorTest, check_netlist_traversal_decorator_shortest_path_to_module) + { + TEST_START + { + std::unique_ptr nl = test_utils::create_empty_netlist(); + ASSERT_NE(nl, nullptr); + Netlist* nl_raw = nl.get(); + const GateLibrary* gl = nl_raw->get_gate_library(); + + // chain of five buffers, the first two in module A, the last two in module B + std::vector g; + for (u32 i = 0; i < 5; i++) + { + g.push_back(nl_raw->create_gate(gl->get_gate_type_by_name("BUF"), "g" + std::to_string(i))); + } + for (u32 i = 0; i + 1 < g.size(); i++) + { + test_utils::connect(nl_raw, g.at(i), "O", g.at(i + 1), "I", "n" + std::to_string(i)); + } + + Module* mod_a = nl_raw->create_module("A", nl_raw->get_top_module(), {g.at(0), g.at(1)}); + Module* mod_b = nl_raw->create_module("B", nl_raw->get_top_module(), {g.at(3), g.at(4)}); + + NetlistTraversalDecorator dec(*nl_raw); + + { + // from the first gate to the far module, stopping at the first gate that belongs to it + auto res = dec.get_shortest_path(g.at(0), mod_b, PinDirection::output); + ASSERT_TRUE(res.is_ok()); + ASSERT_TRUE(res.get().has_value()); + const auto path = res.get().value(); + ASSERT_EQ(path.size(), 4); + EXPECT_EQ(path.front(), g.at(0)); + EXPECT_EQ(path.back(), g.at(3)); + } + { + // a gate that is already inside the module has arrived, so the path is that gate alone + auto res = dec.get_shortest_path(g.at(3), mod_b, PinDirection::output); + ASSERT_TRUE(res.is_ok()); + ASSERT_TRUE(res.get().has_value()); + ASSERT_EQ(res.get().value().size(), 1); + EXPECT_EQ(res.get().value().front(), g.at(3)); + } + { + // between the two modules, from the last gate of A to the first of B + auto res = dec.get_shortest_path(mod_a, mod_b, PinDirection::output); + ASSERT_TRUE(res.is_ok()); + const auto paths = res.get(); + ASSERT_EQ(paths.size(), 1); + EXPECT_EQ(paths.front().front(), g.at(1)); + EXPECT_EQ(paths.front().back(), g.at(3)); + } + { + // the other way round there is no connection + auto res = dec.get_shortest_path(mod_b, mod_a, PinDirection::output); + ASSERT_TRUE(res.is_ok()); + EXPECT_TRUE(res.get().empty()); + } + } + TEST_END + } + + + /** + * Test the traversal that the other traversals of the decorator are written in terms of. + * + * Each case states the same question twice, once through a named traversal and once through the + * core one, so that the claim that they are the same traversal is checked rather than asserted. + * + * Functions: get_gates + */ + TEST_F(DecoratorTest, check_netlist_traversal_decorator_core) + { + TEST_START + { + std::unique_ptr nl = test_utils::create_empty_netlist(); + ASSERT_NE(nl, nullptr); + Netlist* nl_raw = nl.get(); + const GateLibrary* gl = nl_raw->get_gate_library(); + + // ff0 -> inv0 -> inv1 -> ff1, with a second branch inv0 -> inv2 -> ff2 + Gate* ff0 = nl_raw->create_gate(gl->get_gate_type_by_name("DFF"), "ff0"); + Gate* ff1 = nl_raw->create_gate(gl->get_gate_type_by_name("DFF"), "ff1"); + Gate* ff2 = nl_raw->create_gate(gl->get_gate_type_by_name("DFF"), "ff2"); + Gate* inv0 = nl_raw->create_gate(gl->get_gate_type_by_name("INV"), "inv0"); + Gate* inv1 = nl_raw->create_gate(gl->get_gate_type_by_name("INV"), "inv1"); + Gate* inv2 = nl_raw->create_gate(gl->get_gate_type_by_name("INV"), "inv2"); + + test_utils::connect(nl_raw, ff0, "Q", inv0, "I", "n0"); + Net* fan = nl_raw->create_net("fan"); + fan->add_source(inv0, "O"); + fan->add_destination(inv1, "I"); + fan->add_destination(inv2, "I"); + test_utils::connect(nl_raw, inv1, "O", ff1, "D", "n1"); + test_utils::connect(nl_raw, inv2, "O", ff2, "D", "n2"); + + NetlistTraversalDecorator dec(*nl_raw); + const auto is_seq = [](const Gate* g) { return g->get_type()->has_property(GateTypeProperty::sequential); }; + const auto is_comb = [](const Gate* g) { return g->get_type()->has_property(GateTypeProperty::combinational); }; + const auto always = [](const Gate*) { return true; }; + + { + // stopping at a match gives the boundary: the flip-flops behind the logic, and not the logic + auto res = dec.get_gates(ff0, TraversalDirection::forward, is_seq, TraversalStop::at_match); + ASSERT_TRUE(res.is_ok()); + EXPECT_EQ(res.get(), (std::set({ff1, ff2}))); + + auto named = dec.get_next_sequential_gates(ff0, true, {}); + ASSERT_TRUE(named.is_ok()); + EXPECT_EQ(res.get(), named.get()); + } + { + // stopping at a mismatch gives the region: the logic itself, and not the flip-flops + auto res = dec.get_gates(ff0, TraversalDirection::forward, is_comb, TraversalStop::at_mismatch); + ASSERT_TRUE(res.is_ok()); + EXPECT_EQ(res.get(), (std::set({inv0, inv1, inv2}))); + + auto named = dec.get_combinational_cone(ff0, true, {}); + ASSERT_TRUE(named.is_ok()); + EXPECT_EQ(res.get(), named.get()); + } + { + // never stopping walks everything reachable, so both the logic and the flip-flops + auto res = dec.get_gates(ff0, TraversalDirection::forward, always, TraversalStop::never); + ASSERT_TRUE(res.is_ok()); + EXPECT_EQ(res.get(), (std::set({inv0, inv1, inv2, ff1, ff2}))); + } + { + // a depth of one reaches the direct neighbour only + auto res = dec.get_gates(ff0, TraversalDirection::forward, always, TraversalStop::never, 1); + ASSERT_TRUE(res.is_ok()); + EXPECT_EQ(res.get(), (std::set({inv0}))); + } + { + // backwards from a flip-flop at the far end reaches back through the same logic + auto res = dec.get_gates(ff1, TraversalDirection::backward, is_seq, TraversalStop::at_match); + ASSERT_TRUE(res.is_ok()); + EXPECT_EQ(res.get(), (std::set({ff0}))); + } + { + // both directions at once is the union of the two + auto res = dec.get_gates(inv0, TraversalDirection::both, is_seq, TraversalStop::at_match); + ASSERT_TRUE(res.is_ok()); + EXPECT_EQ(res.get(), (std::set({ff0, ff1, ff2}))); + } + { + // a match condition is required, as a traversal that collects nothing says nothing + auto res = dec.get_gates(ff0, TraversalDirection::forward, nullptr, TraversalStop::at_match); + EXPECT_TRUE(res.is_error()); + } + } + TEST_END + } + + + /** + * Test that traversing in both directions returns the union of the two single directions. + * + * The regression this guards: walking both directions from each *adjacent net* also walks forward + * from a fan-in net, which runs into the cones of sibling gates sharing that input -- gates that + * are neither ancestors nor descendants of the start gate. + * + * Functions: get_gates + */ + TEST_F(DecoratorTest, check_netlist_traversal_decorator_both_is_the_union) + { + TEST_START + { + std::unique_ptr nl = test_utils::create_empty_netlist(); + ASSERT_NE(nl, nullptr); + Netlist* nl_raw = nl.get(); + const GateLibrary* gl = nl_raw->get_gate_library(); + + // ff_a drives both the gate under test and a sibling inverter with a flip-flop of its own + Gate* ff_a = nl_raw->create_gate(gl->get_gate_type_by_name("DFF"), "ff_a"); + Gate* ff_b = nl_raw->create_gate(gl->get_gate_type_by_name("DFF"), "ff_b"); + Gate* ff_o = nl_raw->create_gate(gl->get_gate_type_by_name("DFF"), "ff_o"); + Gate* ff_s = nl_raw->create_gate(gl->get_gate_type_by_name("DFF"), "ff_sibling"); + Gate* g = nl_raw->create_gate(gl->get_gate_type_by_name("AND2"), "g"); + Gate* g_s = nl_raw->create_gate(gl->get_gate_type_by_name("INV"), "g_sibling"); + + Net* na = nl_raw->create_net("na"); + na->add_source(ff_a, "Q"); + na->add_destination(g, "I0"); + na->add_destination(g_s, "I"); + test_utils::connect(nl_raw, ff_b, "Q", g, "I1", "nb"); + test_utils::connect(nl_raw, g, "O", ff_o, "D", "no"); + test_utils::connect(nl_raw, g_s, "O", ff_s, "D", "ns"); + + NetlistTraversalDecorator dec(*nl_raw); + const auto is_seq = [](const Gate* gate) { return gate->get_type()->has_property(GateTypeProperty::sequential); }; + + auto res_forward = dec.get_gates(g, TraversalDirection::forward, is_seq, TraversalStop::at_match); + auto res_backward = dec.get_gates(g, TraversalDirection::backward, is_seq, TraversalStop::at_match); + auto res_both = dec.get_gates(g, TraversalDirection::both, is_seq, TraversalStop::at_match); + ASSERT_TRUE(res_forward.is_ok()); + ASSERT_TRUE(res_backward.is_ok()); + ASSERT_TRUE(res_both.is_ok()); + + EXPECT_EQ(res_forward.get(), (std::set({ff_o}))); + EXPECT_EQ(res_backward.get(), (std::set({ff_a, ff_b}))); + + std::set expected = res_forward.get(); + expected.merge(res_backward.get()); + EXPECT_EQ(res_both.get(), expected); + + // the sibling flip-flop belongs to neither direction + EXPECT_EQ(res_both.get().find(ff_s), res_both.get().end()); + } + TEST_END + } + + + /** + * Test that the cached traversals give the same answer as the fresh ones on a netlist with a + * combinational cycle. + * + * The regression this guards: cache entries used to be written while a net was still being + * explored, and a cycle that led the walk back to such a net baked the partial answer into the + * entries of the nets being explored at the time. A later call that reached one of those nets + * through a side path then returned a result with gates missing, which is how the Boolean + * influence plugin produced a wrong dependency matrix on netlists with combinational cycles. + * + * Functions: get_next_sequential_gates, get_combinational_cone, get_next_sequential_gates_map, make_traversal_cache, get_gates + */ + TEST_F(DecoratorTest, check_netlist_traversal_decorator_cache_survives_cycles) + { + TEST_START + { + std::unique_ptr nl = test_utils::create_empty_netlist(); + ASSERT_NE(nl, nullptr); + Netlist* nl_raw = nl.get(); + const GateLibrary* gl = nl_raw->get_gate_library(); + + Gate* ff_first = nl_raw->create_gate(gl->get_gate_type_by_name("DFF"), "ff_first"); + Gate* ff_second = nl_raw->create_gate(gl->get_gate_type_by_name("DFF"), "ff_second"); + Gate* ff_a = nl_raw->create_gate(gl->get_gate_type_by_name("DFF"), "ff_a"); + Gate* ff_b = nl_raw->create_gate(gl->get_gate_type_by_name("DFF"), "ff_b"); + Gate* ff_x = nl_raw->create_gate(gl->get_gate_type_by_name("DFF"), "ff_x"); + Gate* ga = nl_raw->create_gate(gl->get_gate_type_by_name("OR2"), "ga"); + Gate* g_p = nl_raw->create_gate(gl->get_gate_type_by_name("OR2"), "g_p"); + Gate* g_c = nl_raw->create_gate(gl->get_gate_type_by_name("BUF"), "g_c"); + Gate* g_d = nl_raw->create_gate(gl->get_gate_type_by_name("BUF"), "g_d"); + Gate* g_side = nl_raw->create_gate(gl->get_gate_type_by_name("OR2"), "g_side"); + + // the combinational cycle: w0 -> g_d -> f1 -> g_c -> e2 -> g_p -> p -> ga -> w0, + // with a side exit from inside the cycle through g_side into ff_second + Net* w0 = nl_raw->create_net("w0"); + w0->add_source(ga, "O"); + w0->add_destination(ff_first, "D"); + w0->add_destination(g_d, "I"); + Net* p = test_utils::connect(nl_raw, g_p, "O", ga, "I0", "p"); + Net* e3 = test_utils::connect(nl_raw, ff_a, "Q", ga, "I1", "e3"); + Net* e1 = test_utils::connect(nl_raw, ff_b, "Q", g_p, "I0", "e1"); + Net* e2 = nl_raw->create_net("e2"); + e2->add_source(g_c, "O"); + e2->add_destination(g_p, "I1"); + e2->add_destination(g_side, "I1"); + Net* f1 = test_utils::connect(nl_raw, g_d, "O", g_c, "I", "f1"); + Net* x = test_utils::connect(nl_raw, ff_x, "Q", g_side, "I0", "x"); + Net* y = test_utils::connect(nl_raw, g_side, "O", ff_second, "D", "y"); + + NetlistTraversalDecorator dec(*nl_raw); + + const auto is_seq_gate = [](const Gate* g) { return g->get_type()->has_property(GateTypeProperty::sequential); }; + const auto is_comb_gate = [](const Gate* g) { return g->get_type()->has_property(GateTypeProperty::combinational); }; + + { + // one cache shared across two calls, the way the Boolean influence plugin shares one + // across the flip-flops of a netlist + auto cache = dec.make_traversal_cache(TraversalDirection::backward, is_seq_gate, TraversalStop::at_match); + auto first_cached = dec.get_gates(ff_first, cache); + auto second_cached = dec.get_gates(ff_second, cache); + auto first_fresh = dec.get_next_sequential_gates(ff_first, false, {}); + auto second_fresh = dec.get_next_sequential_gates(ff_second, false, {}); + ASSERT_TRUE(first_cached.is_ok()); + ASSERT_TRUE(second_cached.is_ok()); + ASSERT_TRUE(first_fresh.is_ok()); + ASSERT_TRUE(second_fresh.is_ok()); + + EXPECT_EQ(first_fresh.get(), (std::set({ff_a, ff_b}))); + EXPECT_EQ(second_fresh.get(), (std::set({ff_a, ff_b, ff_x}))); + EXPECT_EQ(first_cached.get(), first_fresh.get()); + EXPECT_EQ(second_cached.get(), second_fresh.get()); + } + { + // the combinational region seen through a shared cache covers the whole cycle as well + auto cache = dec.make_traversal_cache(TraversalDirection::backward, is_comb_gate, TraversalStop::at_mismatch); + auto first_cached = dec.get_gates(ff_first, cache); + auto second_cached = dec.get_gates(ff_second, cache); + auto second_fresh = dec.get_combinational_cone(ff_second, false, {}); + ASSERT_TRUE(first_cached.is_ok()); + ASSERT_TRUE(second_cached.is_ok()); + ASSERT_TRUE(second_fresh.is_ok()); + EXPECT_EQ(second_cached.get(), second_fresh.get()); + EXPECT_EQ(second_cached.get(), (std::set({ga, g_p, g_c, g_d, g_side}))); + } + { + // the netlist-wide map shares one cache internally and has to agree with fresh calls + auto map_res = dec.get_next_sequential_gates_map(false, {}); + ASSERT_TRUE(map_res.is_ok()); + for (const auto& [gate, expected] : map_res.get()) + { + auto fresh = dec.get_next_sequential_gates(gate, false, {}); + ASSERT_TRUE(fresh.is_ok()); + EXPECT_EQ(expected, fresh.get()) << "map disagrees with the fresh walk for " << gate->get_name(); + } + } + } + TEST_END + } + + + /** + * Test the sealed traversal cache: results shared across calls agree with the uncached walk, + * and a cache refuses what would make it unsound. + * + * Functions: make_traversal_cache, get_gates + */ + TEST_F(DecoratorTest, check_netlist_traversal_decorator_traversal_cache) + { + TEST_START + { + std::unique_ptr nl = test_utils::create_empty_netlist(); + ASSERT_NE(nl, nullptr); + Netlist* nl_raw = nl.get(); + const GateLibrary* gl = nl_raw->get_gate_library(); + + // ff0 -> inv0 -> {inv1 -> ff1, inv2 -> ff2}, shared logic between the flip-flops + Gate* ff0 = nl_raw->create_gate(gl->get_gate_type_by_name("DFF"), "ff0"); + Gate* ff1 = nl_raw->create_gate(gl->get_gate_type_by_name("DFF"), "ff1"); + Gate* ff2 = nl_raw->create_gate(gl->get_gate_type_by_name("DFF"), "ff2"); + Gate* inv0 = nl_raw->create_gate(gl->get_gate_type_by_name("INV"), "inv0"); + Gate* inv1 = nl_raw->create_gate(gl->get_gate_type_by_name("INV"), "inv1"); + Gate* inv2 = nl_raw->create_gate(gl->get_gate_type_by_name("INV"), "inv2"); + + test_utils::connect(nl_raw, ff0, "Q", inv0, "I", "n0"); + Net* fan = nl_raw->create_net("fan"); + fan->add_source(inv0, "O"); + fan->add_destination(inv1, "I"); + fan->add_destination(inv2, "I"); + test_utils::connect(nl_raw, inv1, "O", ff1, "D", "n1"); + test_utils::connect(nl_raw, inv2, "O", ff2, "D", "n2"); + + NetlistTraversalDecorator dec(*nl_raw); + const auto is_seq = [](const Gate* g) { return g->get_type()->has_property(GateTypeProperty::sequential); }; + + { + // one cache, several starts: every answer equals the uncached walk + auto cache = dec.make_traversal_cache(TraversalDirection::forward, is_seq, TraversalStop::at_match); + for (Gate* start : {ff0, inv0, inv1}) + { + auto cached = dec.get_gates(start, cache); + auto uncached = dec.get_gates(start, TraversalDirection::forward, is_seq, TraversalStop::at_match); + ASSERT_TRUE(cached.is_ok()); + ASSERT_TRUE(uncached.is_ok()); + EXPECT_EQ(cached.get(), uncached.get()) << "cached and uncached disagree from " << start->get_name(); + } + } + { + // a cache seals one direction; both is refused rather than silently mixed + auto cache = dec.make_traversal_cache(TraversalDirection::both, is_seq, TraversalStop::at_match); + EXPECT_TRUE(dec.get_gates(ff0, cache).is_error()); + } + { + // a cache belongs to its netlist + std::unique_ptr other = test_utils::create_empty_netlist(1); + ASSERT_NE(other, nullptr); + auto cache = NetlistTraversalDecorator(*other).make_traversal_cache(TraversalDirection::forward, is_seq, TraversalStop::at_match); + EXPECT_TRUE(dec.get_gates(ff0, cache).is_error()); + } + } + TEST_END + } + } \ No newline at end of file diff --git a/tests/netlist/netlist_utils.cpp b/tests/netlist/netlist_utils.cpp index c038cacf7f91..3ba4d0207cdd 100644 --- a/tests/netlist/netlist_utils.cpp +++ b/tests/netlist/netlist_utils.cpp @@ -1,3 +1,4 @@ +#include "hal_core/netlist/decorators/netlist_traversal_decorator.h" #include "hal_core/netlist/netlist_utils.h" #include "gate_library_test_utils.h" @@ -584,9 +585,13 @@ namespace hal Net* common_net2 = test_utils::connect(nl.get(), l1, "O", l2, "I1"); test_utils::connect(nl.get(), l1, "O", l3, "I1"); - std::vector gates = {l2, l3, l4, l5}; - std::vector common_nets4 = netlist_utils::get_common_inputs(gates); - std::vector common_nets2 = netlist_utils::get_common_inputs(gates, 2); + std::vector gates = {l2, l3, l4, l5}; + auto res_common_nets4 = NetlistTraversalDecorator(*nl).get_common_inputs(gates); + auto res_common_nets2 = NetlistTraversalDecorator(*nl).get_common_inputs(gates, 2); + ASSERT_TRUE(res_common_nets4.is_ok()); + ASSERT_TRUE(res_common_nets2.is_ok()); + std::vector common_nets4 = res_common_nets4.get(); + std::vector common_nets2 = res_common_nets2.get(); ASSERT_EQ(common_nets4.size(), 1); EXPECT_TRUE(std::find(common_nets4.begin(), common_nets4.end(), common_net4) != common_nets4.end()); @@ -653,35 +658,35 @@ namespace hal std::vector expected_chain = {c0, c1, c2, c3}; { - auto res = netlist_utils::get_gate_chain(c0, {carry->get_pin_by_name("CI")}, {carry->get_pin_by_name("CO")}); + auto res = NetlistTraversalDecorator(*nl).get_gate_chain(c0, {carry->get_pin_by_name("CI")}, {carry->get_pin_by_name("CO")}); ASSERT_TRUE(res.is_ok()); std::vector chain = res.get(); EXPECT_EQ(chain, expected_chain); } { - auto res = netlist_utils::get_gate_chain(c1, {carry->get_pin_by_name("CI")}, {carry->get_pin_by_name("CO")}); + auto res = NetlistTraversalDecorator(*nl).get_gate_chain(c1, {carry->get_pin_by_name("CI")}, {carry->get_pin_by_name("CO")}); ASSERT_TRUE(res.is_ok()); std::vector chain = res.get(); EXPECT_EQ(chain, expected_chain); } { - auto res = netlist_utils::get_gate_chain(c2, {carry->get_pin_by_name("CI")}, {carry->get_pin_by_name("CO")}); + auto res = NetlistTraversalDecorator(*nl).get_gate_chain(c2, {carry->get_pin_by_name("CI")}, {carry->get_pin_by_name("CO")}); ASSERT_TRUE(res.is_ok()); std::vector chain = res.get(); EXPECT_EQ(chain, expected_chain); } { - auto res = netlist_utils::get_gate_chain(c3, {carry->get_pin_by_name("CI")}, {carry->get_pin_by_name("CO")}); + auto res = NetlistTraversalDecorator(*nl).get_gate_chain(c3, {carry->get_pin_by_name("CI")}, {carry->get_pin_by_name("CO")}); ASSERT_TRUE(res.is_ok()); std::vector chain = res.get(); EXPECT_EQ(chain, expected_chain); } { - auto res = netlist_utils::get_gate_chain(g0, {carry->get_pin_by_name("CI")}, {carry->get_pin_by_name("CO")}, [](const Gate* g) { + auto res = NetlistTraversalDecorator(*nl).get_gate_chain(g0, {carry->get_pin_by_name("CI")}, {carry->get_pin_by_name("CO")}, [](const Gate* g) { return g->get_type()->has_property(GateTypeProperty::c_carry); }); ASSERT_TRUE(res.is_error()); @@ -715,25 +720,25 @@ namespace hal std::vector expected_chain = {c0, c1, c2, c3}; { - auto res = netlist_utils::get_gate_chain(c0); + auto res = NetlistTraversalDecorator(*nl).get_gate_chain(c0); ASSERT_TRUE(res.is_ok()); std::vector chain = res.get(); EXPECT_EQ(chain, expected_chain); } { - auto res = netlist_utils::get_gate_chain(c1); + auto res = NetlistTraversalDecorator(*nl).get_gate_chain(c1); ASSERT_TRUE(res.is_ok()); std::vector chain = res.get(); EXPECT_EQ(chain, expected_chain); } { - auto res = netlist_utils::get_gate_chain(c2); + auto res = NetlistTraversalDecorator(*nl).get_gate_chain(c2); ASSERT_TRUE(res.is_ok()); std::vector chain = res.get(); EXPECT_EQ(chain, expected_chain); } { - auto res = netlist_utils::get_gate_chain(c3); + auto res = NetlistTraversalDecorator(*nl).get_gate_chain(c3); ASSERT_TRUE(res.is_ok()); std::vector chain = res.get(); EXPECT_EQ(chain, expected_chain); @@ -759,7 +764,7 @@ namespace hal test_utils::connect(nl.get(), c3, "CO", c0, "CI"); { - auto res = netlist_utils::get_gate_chain(c0); + auto res = NetlistTraversalDecorator(*nl).get_gate_chain(c0); ASSERT_TRUE(res.is_ok()); std::vector chain = res.get(); EXPECT_EQ(chain, std::vector({c0, c1, c2, c3})); @@ -788,37 +793,37 @@ namespace hal test_utils::connect(nl.get(), c4, "CO", c5, "CI"); { - auto res = netlist_utils::get_gate_chain(c0); + auto res = NetlistTraversalDecorator(*nl).get_gate_chain(c0); ASSERT_TRUE(res.is_ok()); std::vector chain = res.get(); EXPECT_EQ(chain, std::vector({c0, c1})); } { - auto res = netlist_utils::get_gate_chain(c1); + auto res = NetlistTraversalDecorator(*nl).get_gate_chain(c1); ASSERT_TRUE(res.is_ok()); std::vector chain = res.get(); EXPECT_EQ(chain, std::vector({c0, c1})); } { - auto res = netlist_utils::get_gate_chain(c2); + auto res = NetlistTraversalDecorator(*nl).get_gate_chain(c2); ASSERT_TRUE(res.is_ok()); std::vector chain = res.get(); EXPECT_EQ(chain, std::vector({c0, c1, c2, c3})); } { - auto res = netlist_utils::get_gate_chain(c3); + auto res = NetlistTraversalDecorator(*nl).get_gate_chain(c3); ASSERT_TRUE(res.is_ok()); std::vector chain = res.get(); EXPECT_EQ(chain, std::vector({c0, c1, c2, c3})); } { - auto res = netlist_utils::get_gate_chain(c4); + auto res = NetlistTraversalDecorator(*nl).get_gate_chain(c4); ASSERT_TRUE(res.is_ok()); std::vector chain = res.get(); EXPECT_EQ(chain, std::vector({c0, c1, c4, c5})); } { - auto res = netlist_utils::get_gate_chain(c5); + auto res = NetlistTraversalDecorator(*nl).get_gate_chain(c5); ASSERT_TRUE(res.is_ok()); std::vector chain = res.get(); EXPECT_EQ(chain, std::vector({c0, c1, c4, c5})); diff --git a/tests/python_bindings/CMakeLists.txt b/tests/python_bindings/CMakeLists.txt index 9104556082c4..2b9b5a06e5c3 100644 --- a/tests/python_bindings/CMakeLists.txt +++ b/tests/python_bindings/CMakeLists.txt @@ -12,3 +12,12 @@ add_test(NAME runTest-python_binding_smoke WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}) set_tests_properties(runTest-python_binding_smoke PROPERTIES ENVIRONMENT "PYTHONPATH=${CMAKE_BINARY_DIR}/lib;HAL_BASE_PATH=${CMAKE_BINARY_DIR}") + +# Loading the plugins and letting the interpreter exit without unloading them used to segfault, so +# this checks that a process which does exactly that comes back with a zero exit code. It runs its +# cases in subprocesses, as what is under test is how a process ends. +add_test(NAME runTest-python_teardown + COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/teardown_test.py + WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}) +set_tests_properties(runTest-python_teardown PROPERTIES + ENVIRONMENT "PYTHONPATH=${CMAKE_BINARY_DIR}/lib;HAL_BASE_PATH=${CMAKE_BINARY_DIR}") diff --git a/tests/python_bindings/check_binding_lifetimes.py b/tests/python_bindings/check_binding_lifetimes.py index e5a1ebde9087..dce7401af83f 100644 --- a/tests/python_bindings/check_binding_lifetimes.py +++ b/tests/python_bindings/check_binding_lifetimes.py @@ -81,8 +81,10 @@ def main() -> int: submodules = set(re.findall(r"auto\s+(\w+)\s*=\s*\w+\.def_submodule", text)) for match in DEF_RE.finditer(text): receiver, form, name = match.groups() - if receiver in submodules or receiver == "m" or form == "def_static": - continue # no receiver to tie the result to, see the deferred cases + # Free, static and submodule-level functions used to be exempt because nothing could fix + # them: keep_alive cannot nurse a returned list and reference_internal has no parent + # there. hal::borrowed() has neither problem -- it resolves each returned object's owner + # through the owner's own wrapper -- so they are held to the same rule as methods. end = text.find(')");', match.start()) body = text[match.start(): end if end != -1 else match.start() + 4000] if 'R"(' not in body: diff --git a/tests/python_bindings/smoke_test_bindings.py b/tests/python_bindings/smoke_test_bindings.py index 46f0a205a453..7a61373a78ec 100644 --- a/tests/python_bindings/smoke_test_bindings.py +++ b/tests/python_bindings/smoke_test_bindings.py @@ -17,6 +17,7 @@ that looks exactly like a real failure. """ +import faulthandler import os import re import sys @@ -172,15 +173,12 @@ def main(): import hal_py # The gate library is read by a plugin, so the plugins have to be loaded before anything else. - # They also have to be unloaded again before the interpreter exits: leaving them loaded segfaults - # at teardown, independently of anything this test does. That is a real defect, but it predates - # this test -- it reproduces on a build from well before the binding work -- so it is not this - # test's job to fail on it. + # They are left loaded, which used to segfault at teardown. This test cannot be relied on to + # notice if that comes back, though, because it imports the plugin modules as well and an + # imported module holds a reference of its own to the library, which keeps it mapped. See + # teardown_test.py, which reproduces it without importing anything. hal_py.plugin_manager.load_all_plugins() - try: - return run_checks(hal_py) - finally: - hal_py.plugin_manager.unload_all_plugins() + return run_checks(hal_py) def run_checks(hal_py): @@ -188,7 +186,7 @@ def run_checks(hal_py): called = [] imported = import_plugin_modules(failures) - print(f"imported {len(imported)} plugin modules") + print(f"imported {len(imported)} plugin modules", flush=True) objects = build_netlist(hal_py) for label, obj in objects.items(): @@ -226,4 +224,7 @@ def run_checks(hal_py): if __name__ == "__main__": + # A crash inside a binding would otherwise leave no trace of which binding it was: the test's own + # output is block-buffered when ctest captures it. This prints the Python stack on a fatal signal. + faulthandler.enable() sys.exit(main()) diff --git a/tests/python_bindings/teardown_test.py b/tests/python_bindings/teardown_test.py new file mode 100644 index 000000000000..f75058794ebc --- /dev/null +++ b/tests/python_bindings/teardown_test.py @@ -0,0 +1,52 @@ +#!/usr/bin/env python3 +"""Check that an interpreter which loaded the plugins can exit. + +Loading the plugins and letting the interpreter exit without unloading them used to segfault. The +parser and writer registries live in libhal_netlist and hold a std::function per plugin that provides +one; the plugin libraries were closed when the map holding them was destroyed, which happens before +those registries are, so every one of those functions was left pointing into memory that had been +unmapped. + +The check runs in a subprocess, because what is being tested is the exit of a process rather than +anything observable from within it. It also has to avoid importing any plugin module: an imported +module holds a reference of its own to the library, which keeps it mapped and hides the fault. That +is why the binding smoke test does not notice this one. +""" + +import os +import subprocess +import sys + +CASES = { + "load the plugins and exit": "import hal_py; hal_py.plugin_manager.load_all_plugins()", + "load, unload, and exit": "import hal_py; hal_py.plugin_manager.load_all_plugins(); hal_py.plugin_manager.unload_all_plugins()", + "load, unload, load again, and exit": ( + "import hal_py; " + "hal_py.plugin_manager.load_all_plugins(); " + "hal_py.plugin_manager.unload_all_plugins(); " + "hal_py.plugin_manager.load_all_plugins()" + ), +} + + +def main(): + failures = [] + for description, code in CASES.items(): + result = subprocess.run([sys.executable, "-c", code], capture_output=True, env=os.environ.copy()) + if result.returncode != 0: + reason = f"signal {-result.returncode}" if result.returncode < 0 else f"exit code {result.returncode}" + failures.append((description, reason, result.stderr.decode(errors="replace")[-400:])) + print(f" {description}: {'ok' if result.returncode == 0 else 'FAILED'}") + + if failures: + print(f"\n{len(failures)} interpreter(s) did not exit cleanly:\n", file=sys.stderr) + for description, reason, stderr in failures: + print(f"--- {description}: {reason} ---", file=sys.stderr) + print(stderr, file=sys.stderr) + return 1 + + return 0 + + +if __name__ == "__main__": + sys.exit(main())