{"id":506,"date":"2026-03-04T07:39:46","date_gmt":"2026-03-03T23:39:46","guid":{"rendered":"https:\/\/connectword.dpdns.org\/?p=506"},"modified":"2026-03-04T07:39:46","modified_gmt":"2026-03-03T23:39:46","slug":"meet-symtorch-a-pytorch-library-that-translates-deep-learning-models-into-human-readable-equations","status":"publish","type":"post","link":"https:\/\/connectword.dpdns.org\/?p=506","title":{"rendered":"Meet SymTorch: A PyTorch Library that Translates Deep Learning Models into Human-Readable Equations"},"content":{"rendered":"<p>Can symbolic regression be the key to transforming opaque deep learning models into interpretable, closed-form mathematical equations? or Say you have trained your deep learning model. It works. But do you know what it has actually learned? A team of University of Cambridge researchers propose \u2018SymTorch\u2019, a library designed to integrate <strong>symbolic regression (SR)<\/strong> into deep learning workflows. It enables researchers to approximate neural network components with closed-form mathematical expressions, facilitating functional interpretability and potential inference acceleration.<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1638\" height=\"922\" data-attachment-id=\"78197\" data-permalink=\"https:\/\/www.marktechpost.com\/2026\/03\/03\/meet-symtorch-a-pytorch-library-that-translates-deep-learning-models-into-human-readable-equations\/screenshot-2026-03-03-at-3-39-17-pm-2\/\" data-orig-file=\"https:\/\/www.marktechpost.com\/wp-content\/uploads\/2026\/03\/Screenshot-2026-03-03-at-3.39.17-PM-1.png\" data-orig-size=\"1638,922\" data-comments-opened=\"1\" data-image-meta='{\"aperture\":\"0\",\"credit\":\"\",\"camera\":\"\",\"caption\":\"\",\"created_timestamp\":\"0\",\"copyright\":\"\",\"focal_length\":\"0\",\"iso\":\"0\",\"shutter_speed\":\"0\",\"title\":\"\",\"orientation\":\"0\"}' data-image-title=\"Screenshot 2026-03-03 at 3.39.17\u202fPM\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/www.marktechpost.com\/wp-content\/uploads\/2026\/03\/Screenshot-2026-03-03-at-3.39.17-PM-1-300x169.png\" data-large-file=\"https:\/\/www.marktechpost.com\/wp-content\/uploads\/2026\/03\/Screenshot-2026-03-03-at-3.39.17-PM-1-1024x576.png\" src=\"https:\/\/www.marktechpost.com\/wp-content\/uploads\/2026\/03\/Screenshot-2026-03-03-at-3.39.17-PM-1.png\" alt=\"\" class=\"wp-image-78197\" \/><figcaption class=\"wp-element-caption\">https:\/\/arxiv.org\/pdf\/2602.21307<\/figcaption><\/figure>\n<\/div>\n<h3 class=\"wp-block-heading\"><strong>Core Mechanism: The Wrap-Distill-Switch Workflow<\/strong><\/h3>\n<p>SymTorch simplifies the engineering required to extract symbolic equations from trained models by automating data movement and hook management.<\/p>\n<ul class=\"wp-block-list\">\n<li><strong>Wrap:<\/strong> Users apply the <code>SymbolicModel<\/code> wrapper to any <code>nn.Module<\/code> or callable function.<\/li>\n<li><strong>Distill:<\/strong> The library registers forward hooks to record input and output activations during a forward pass. These are cached and transferred from the GPU to the CPU for symbolic regression via PySR.<\/li>\n<li><strong>Switch:<\/strong> Once distilled, the original neural weights can be replaced with the discovered equation in the forward pass using <code>switch_to_symbolic<\/code>.<\/li>\n<\/ul>\n<p>The library interfaces with <strong>PySR<\/strong>, which uses a multi-population genetic algorithm to find equations that balance accuracy and complexity on a <strong>Pareto front<\/strong>. The \u2018best\u2019 equation is chosen by maximizing the fractional drop in log mean absolute error relative to an increase in complexity.<\/p>\n<h3 class=\"wp-block-heading\"><strong>Case Study: Accelerating LLM Inference<\/strong><\/h3>\n<p>A primary application explored in this research is replacing <strong>Multi-Layer Perceptron (MLP)<\/strong> layers in Transformer models with symbolic surrogates to improve throughput.<\/p>\n<h3 class=\"wp-block-heading\"><strong>Implementation Details<\/strong><\/h3>\n<p>Due to the high dimensionality of LLM activations, the research team employed <strong>Principal Component Analysis (PCA)<\/strong> to compress inputs and outputs before performing SR. For the <strong>Qwen2.5-1.5B<\/strong> model, they selected 32 principal components for inputs and 8 for outputs across three targeted layers.<\/p>\n<h3 class=\"wp-block-heading\"><strong>Performance Trade-offs<\/strong><\/h3>\n<p>The intervention resulted in an <strong>8.3% increase in token throughput<\/strong><sup><\/sup><sup><\/sup><sup><\/sup><sup><\/sup>. However, this gain came with a non-trivial increase in perplexity, primarily driven by the PCA dimensionality reduction rather than the symbolic approximation itself<sup><\/sup>.<\/p>\n<figure class=\"wp-block-table is-style-stripes\">\n<table class=\"has-fixed-layout\">\n<thead>\n<tr>\n<td><strong>Metric<\/strong><\/td>\n<td><strong>Baseline (Qwen2.5-1.5B)<\/strong><\/td>\n<td><strong>Symbolic Surrogate<\/strong><\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td><strong>Perplexity (Wikitext-2)<\/strong><\/td>\n<td>10.62<\/td>\n<td>13.76<\/td>\n<\/tr>\n<tr>\n<td><strong>Throughput (tokens\/s)<\/strong><\/td>\n<td>4878.82<\/td>\n<td>5281.42<\/td>\n<\/tr>\n<tr>\n<td><strong>Avg. Latency (ms)<\/strong><\/td>\n<td>209.89<\/td>\n<td>193.89<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/figure>\n<h3 class=\"wp-block-heading\"><strong>GNNs and PINNs<\/strong><\/h3>\n<p>SymTorch was validated on its ability to recover known physical laws from latent representations in scientific models<sup><\/sup><sup><\/sup><sup><\/sup><sup><\/sup>.<\/p>\n<ul class=\"wp-block-list\">\n<li><strong>Graph Neural Networks (GNNs):<\/strong> By training a GNN on particle dynamics, the research team used SymTorch to recover empirical force laws, such as gravity (1\/r<sup>2<\/sup>) and spring forces, directly from the edge messages.<\/li>\n<li><strong>Physics-Informed Neural Networks (PINNs):<\/strong> The library successfully distilled the 1-D heat equation\u2019s analytic solution from a trained PINN. The PINN\u2019s inductive bias allowed it to achieve a Mean Squared Error (MSE) of 7.40 x 10<sup>-6<\/sup>.<\/li>\n<li><strong>LLM Arithmetic Analysis:<\/strong> Symbolic distillation was used to inspect how models like Llama-3.2-1B perform 3-digit addition and multiplication. The distilled equations revealed that while the models are often correct, they rely on internal heuristics that include systematic numerical errors.<\/li>\n<\/ul>\n<h3 class=\"wp-block-heading\"><strong>Key Takeaways<\/strong><\/h3>\n<ul class=\"wp-block-list\">\n<li><strong>Automated Symbolic Distillation<\/strong>: SymTorch is a library that automates the process of replacing complex neural network components with interpretable, closed-form mathematical equations by wrapping components and collecting their input-output behavior.<\/li>\n<li><strong>Engineering Barrier Removal<\/strong>: The library handles critical engineering challenges that previously hindered the adoption of symbolic regression, including GPU-CPU data transfer, input-output caching, and seamless switching between neural and symbolic forward passes.<\/li>\n<li><strong>LLM Inference Acceleration<\/strong>: A proof-of-concept demonstrated that replacing MLP layers in a transformer model with symbolic surrogates achieved an 8.3% throughput improvement, though with some performance degradation in perplexity.<\/li>\n<li><strong>Scientific Law Discovery<\/strong>: SymTorch was successfully used to recover physical laws from Graph Neural Networks (GNNs) and analytic solutions to the 1-D heat equation from Physics-Informed Neural Networks (PINNs).<\/li>\n<li><strong>Functional Interpretability of LLMs<\/strong>: By distilling the end-to-end behavior of LLMs, researchers could inspect the explicit mathematical heuristics used for tasks like arithmetic, revealing where internal logic deviates from exact operations.<\/li>\n<\/ul>\n<hr class=\"wp-block-separator has-alpha-channel-opacity\" \/>\n<p>Check out the\u00a0<strong><a href=\"https:\/\/arxiv.org\/pdf\/2602.21307\" target=\"_blank\" rel=\"noreferrer noopener\">Paper<\/a>, <a href=\"https:\/\/github.com\/astroautomata\/SymTorch\" target=\"_blank\" rel=\"noreferrer noopener\">Repo<\/a> <\/strong>and<strong><a href=\"https:\/\/astroautomata.github.io\/symtorch-web\/\" target=\"_blank\" rel=\"noreferrer noopener\"> Project Page<\/a>.\u00a0<\/strong>Also,\u00a0feel free to follow us on\u00a0<strong><a href=\"https:\/\/x.com\/intent\/follow?screen_name=marktechpost\" target=\"_blank\" rel=\"noreferrer noopener\"><mark>Twitter<\/mark><\/a><\/strong>\u00a0and don\u2019t forget to join our\u00a0<strong><a href=\"https:\/\/www.reddit.com\/r\/machinelearningnews\/\" target=\"_blank\" rel=\"noreferrer noopener\">120k+ ML SubReddit<\/a><\/strong>\u00a0and Subscribe to\u00a0<strong><a href=\"https:\/\/www.aidevsignals.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">our Newsletter<\/a><\/strong>. Wait! are you on telegram?\u00a0<strong><a href=\"https:\/\/t.me\/machinelearningresearchnews\" target=\"_blank\" rel=\"noreferrer noopener\">now you can join us on telegram as well.<\/a><\/strong><\/p>\n<p>The post <a href=\"https:\/\/www.marktechpost.com\/2026\/03\/03\/meet-symtorch-a-pytorch-library-that-translates-deep-learning-models-into-human-readable-equations\/\">Meet SymTorch: A PyTorch Library that Translates Deep Learning Models into Human-Readable Equations<\/a> appeared first on <a href=\"https:\/\/www.marktechpost.com\/\">MarkTechPost<\/a>.<\/p>","protected":false},"excerpt":{"rendered":"<p>Can symbolic regression be the&hellip;<\/p>\n","protected":false},"author":1,"featured_media":507,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-506","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/posts\/506","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=506"}],"version-history":[{"count":0,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/posts\/506\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/media\/507"}],"wp:attachment":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=506"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=506"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=506"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}