Skydio Researchers Open-Source ‘SymForce’: A Fast Symbolic Computation And Code Generation Library For Robotics Applications Like Computer Vision, etc.

This Article is written as a summay by Marktechpost Research Staff based on the research paper 'SymForce: Symbolic Computation and
Code Generation for Robotics'. All Credit For This Research Goes To Apple Researchers on This Project. Checkout the paper and github.

Please Don't Forget To Join Our ML Subreddit

SymForce is a library for symbolic computation and code generation. It allows to code a problem in python, perform symbolic experimentation, generate optimized code and execute extremely competent optimization problems that depend on the original problem definition. Figure 1 depicts that Symforce outperforms sparse and dense matrix multiplication by utilizing sparsity and sharing common subexpressions with no runtime overhead.


This exceptional contribution of this research is to provide a free and open-source library having symbolic implementations of geometry and camera types with Lie group operations, as well as fast runtime classes with identical interfaces. It also generates code for turning arbitrary symbolic functions into structured and quick runtime functions. 

The architecture of Symforce is represented in figure 2. It showcases the primary components of SymForce, such as symbolic computation, code generation, and optimization.


Symbolic Computation: Symforce extends SymPy API and provides tools to build and analyze complex symbolic expressions in python. Symforce supports two symbolic backends – SymPy and SymEngine. Symforce implements several core types used in robotics, such as matrices, poses, rotations, barrier functions, camera models, and noise models. This approach inspired by GTSAM employs a concepts mechanism instead of inheritance to allow generic code to function on any geometry type. StorageOps, GroupOps, and LiegroupOps are the three main concepts utilized in Symforce. This approach uses codegen class that results in the native code being flattened yet with a structured and human-readable interface that works well with geometric types and optimization framework.

Sparsity Exploitation: Symforce resolves the multiplication of matrices with zero entries by providing order of magnitude speedups.

Algebraic Simplification: Symbolic expressions can be algebraically converted into computationally fast forms. This can be achieved using expansion, factorization, term collection, cancellation, fraction decomposition, trigonometric and logarithmic identities, series expansions, and limits.

Symbolic Differentiation: Computing tangent-space derivatives of any user-defined functions working on Lie group types, which is required for on-manifold optimization and uncertainty propagation, is one of SymForce’s most important capabilities. Symbolic differentiation has several advantages over Automatic Differentiation (AD) for computing tangent space jacobians, requiring less handwritten code. Also, it shares more subexpressions as well as eliminates the need for matrix multiplication at runtime. Also, it avoids dynamic memory allocation and dense chain ruling at runtime. In contrast, automatic differentiation is utilized for computing derivatives of large computation graphs.

The proposed approach is compared with Eigen for sparse and dense matrix multiplication; GTSAM, Sophus, and JAX for tangent space differentiation; and GTSAM, Ceres, and JAX for nonlinear least-squares. This approach measures CPU time, instruction counts, and L1 cache loads on an Intel i7 CPU and NVIDIA Tegra X2 ARM CPU. Execution time is measured in JAX on an RTX 2080 Ti GPU. 

Two experiments that are performed include the matrix multiplication experiment and the inverse compose experiment. The matrix multiplication experiment shows that the flattened function outperforms the dense fixed approach in all categories. The results also show that Dynamic memory allocation is a poor tradeoff for small matrices as it consumes most of the processing time. The proposed approach outperforms the second-best by 8.7x on Intel and 11.6x on Tegra for the matrix n3c4_b2. In the inverse compose experiment, the previous idea is extended for computing tangent space jacobians. SymForce is used to generate Jacobian functions for the inverse and compose functions independently. Finally, a practical example is demonstrated to estimate a robot trajectory utilizing point measurements. 

The proposed approach of flattening expressions becomes impractical if there are heavily nested expressions, such as long chains of integrations or loops. The separation between symbolic and runtime contexts needs higher-level abstraction than implementing runtime code directly. Large expressions slow down several symbolic routines, such as simplification and factorization. The utilized optimizer does not support hard constraints or custom solvers.

SymForce and its underlying techniques were described in this study as a powerful strategy for solving various computing challenges. It results in faster runtime Code and requires minimal development time and fewer lines of handwritten code compared to its state-of-the-art techniques. Its auto-generating code is a prime advantage that flattens across function calls and matrix multiplications.

🐝 Join the Fastest Growing AI Research Newsletter Read by Researchers from Google + NVIDIA + Meta + Stanford + MIT + Microsoft and many others...