Deepsolute LLC | Deep Learning Optimization & AI Consulting

About

Research depth, delivered as working systems

Deepsolute LLC is an AI engineering firm focused on making deep learning efficient enough to run where it matters: in real-time trading systems, on long sensor and transaction streams, and inside enterprise products with real cost and latency budgets.

Our work runs from the math of attention down to the GPU kernel. We design architectures, write the CUDA, profile it, and ship it. Clients get a model that is faster and cheaper to run, plus a team that can explain why.

Services

What we do

Focused engagements, from a short architecture review to building and shipping a full system.

Low-latency inference

Fused kernels, memory-aware layouts, and serving stacks (vLLM, Triton, ONNX Runtime) tuned to hit hard latency and throughput targets.

Sparse & efficient attention

Sliding-window, dilated, and learned sparsity patterns that take transformers from O(n²) to near-linear cost on long sequences.

Time series foundation models

Forecasting, anomaly detection, and signal extraction on market, sensor, and transaction data, built on modern TSFM architectures.

Custom CUDA development

C++/CUDA kernels and PyTorch extensions for the hot paths that general-purpose frameworks leave on the table, profiled with Nsight.

Agentic & generative AI

Multi-agent orchestration, advanced RAG, knowledge graphs, and fine-tuned open-weight LLMs deployed across AWS, Azure, and GCP.

Architecture & strategy

Independent reviews of model design, infrastructure, and cost. You get a clear plan for what to build, what to buy, and what to skip.

Model training

Fine-tuning and pretraining for open-weight models

Off-the-shelf models know the internet. We teach them your domain, on your infrastructure, with weights you own. Most engagements start with fine-tuning and move to pretraining only when the data and the results justify it.

Adapt

Parameter-efficient fine-tuning

LoRA and QLoRA adapters train a small fraction of the parameters. That keeps GPU budgets modest, lets one base model serve many tasks, and makes fast iteration practical.

SFT: Supervised fine-tuning on curated task data: extraction, structured outputs, domain Q&A, and tool use.
DPO: Direct preference optimization on pairs ranked by your experts, so the model prefers answers your specialists would sign off on.
Ship: Evaluation harnesses, adapter merging, quantization, and serving on vLLM or Triton.

Extend

Continued & domain pretraining

When fine-tuning hits the limits of what the base model has seen, we extend the base model itself. Continued pretraining on a large in-domain corpus teaches the vocabulary, notation, and structure of your field.

Data: Corpus sourcing, cleaning, de-duplication, tokenizer review, and domain/general data mixing to avoid forgetting.
Train: Distributed training with FSDP or DeepSpeed, mixed precision, checkpointing, and kernel-level performance tuning.
Models: Language and vision-language models, from open-weight families such as Llama, Qwen, and Mistral.

Case studies

Selected client work

Two engagements, one theme: getting state-of-the-art models to work on data they were never built for.

Quantitative tradingSparse attention

Hours of market context at tick-level speed

Sliding-window sparse attention and custom CUDA kernels let a trading firm's models see a 64K-tick window, cutting attention memory from 64.2 GB to 0.31 GB.

Read the case study →

EngineeringVision-language models

Teaching vision-language models to read engineering drawings

Phase 1 PEFT fine-tuning and Phase 2 continued pretraining of open-weight VLMs on raster and vector drawing PDFs, using a layered approach borrowed from EDA.

Read the case study →

Featured research

MA-Transformer

A microstructure-aware transformer for order book imbalance prediction in high-frequency markets. It is open source under Apache 2.0.

Dense self-attention scales quadratically with sequence length. On tick data that quickly becomes the bottleneck: a few hours of a trading session will not fit in GPU memory, let alone run in real time.

MA-Transformer restricts each query to a sliding window of recent ticks, which exploits the temporal locality of market data and cuts attention cost from O(n²) to O(n·w). The performance-critical path is written in custom C++/CUDA:

Sparse self-attention kernels tuned for financial time series
GPU-native microstructure features (VWAP, order book imbalance, realized volatility)
Fused QKV, attention, and feed-forward operations to cut kernel launches
Circular-buffer KV caches for continuous streaming inference

View the project on GitHub →

At 64K-tick sequences (about 3–4 hours of tick data)

	Dense	Sparse
Memory	64.2 GB	0.31 GB
Compute	4,398 GFLOPs	4.3 GFLOPs
Complexity	O(n²)	O(n·w)
16 GB GPU	Out of memory	Runs

Theoretical estimates: fp32, batch 1, four heads, head dim 64, window 64; dense assumes the full score matrix is stored (16 GB per head). Details in the whitepaper below.

Whitepaper

The Theory and Mathematics of Sparse Attention Mechanisms

Timothy M. Fox · October 2026 · 7 pages

A rigorous treatment of sparse attention: how masked softmax over a sparsity set reduces cost from O(n²) to near-linear, and the trade-offs of sliding-window, causal, dilated, block-sparse, global, and fixed patterns. It introduces a causal local-plus-dilated pattern for tick data and covers recent trainable sparsity such as NSA and MoBA.

Download our whitepaper LaTeX source on GitHub →

From our writing

The “Quadratic Tax”: why financial foundation models struggle to scale

Timothy M. Fox · December 17, 2025 · LinkedIn

The NYSE now processes more than 1.2 trillion messages a day, and catching a regime shift or volatility spike can mean looking back across 64,000+ ticks. Dense attention charges a quadratic tax on that context: double the sequence and cost quadruples.

The article makes the case for sub-quadratic attention built on hardware-aware sparsity. It covers sliding windows for local microstructure and dilated patterns for longer trends, implemented as fused, memory-coalesced CUDA kernels so near-linear scaling shows up as real throughput on NVIDIA Hopper and Blackwell GPUs.

The goal isn’t just to make models “faster.” The goal is to make them economically scalable.

Read the article Discuss on LinkedIn →

Leadership

The team

Timothy Fox

Principal & Lead AI Architect

Tim has spent more than two decades building machine learning and large-scale data systems for enterprises in finance, retail, healthcare, media, and energy. His work spans the stack, from custom CUDA kernels and sparse attention research to multi-agent LLM systems, advanced RAG, and knowledge-graph reasoning running in production on AWS, Azure, and GCP.

He has served as Lead Architect at Scaia and spent years as a scalable AI consultant with CreaTek Solutions, where he designed platforms ranging from 450-node Hadoop clusters to Kubernetes-based model serving. He is a LinkedIn Learning instructor (Data Science with Python), holds Google Cloud Data Engineer and Cloud Architect certifications, and earned a BS in Computer Engineering from Virginia Tech.

Sparse attention
CUDA
Time series
LLMs & agents
MLOps

LinkedIn GitHub

Bruce Barker

Principal & Chief Operating Officer

Bruce is a veteran of many startups and leads Deepsolute's operations, client engagements, and business development. He makes sure the technical work turns into delivered results on schedule.

Contact

Have a model that needs to be faster?

We are taking on new consulting engagements. Tell us what you are building and where it is stuck, and we will reply within two business days.

contact@deepsolute.com

Deepsolute LLC 1530 PB Ln F4201 Wichita Falls, TX 76302 Timothy Fox on LinkedIn →

Faster models.Longer context.Production-ready AI.