Showing posts with label autonomous driving. Show all posts
Showing posts with label autonomous driving. Show all posts

Sunday, July 26, 2026

Tesla FSD Technical Analysis 2 - Data, OTA and the Safety Test

Series · Tesla FSD Technical Analysis

Article series · Completed

Episode 2 · Tesla FSD Technical Analysis 2 - Data, OTA and the Safety Test

Tesla FSD Technical Analysis 2 - Data, OTA and the Safety Test

This series tests Tesla FSD's implementation against public sources. Part 1 established that driving inference happens inside the vehicle. That raises the next questions: Does Tesla upload continuous video from millions of cars? Who pays for connectivity? How can OTA replace a large AI model? And is the architecture objectively advanced enough to deserve its reputation?

Does every video go to a server: how fleet learning works

The short answer is selective collection, not continuous raw streaming. Uploading every high-resolution camera for an entire trip would make cellular and storage costs prohibitive. Tesla has repeatedly described local processing followed by transmission of short clips or telemetry selected for particular safety events and learning conditions.

Tesla's rideshare privacy notice says external-camera ADAS processing stays in the vehicle by default. It describes two classes of transmissible recordings: safety events and fleet learning. Safety-event clips can be up to 30 seconds and may be linked with account, time and location metadata after a collision, airbag deployment or emergency stop. Fleet-learning clips help recognize lanes, signs and traffic lights and are described as not linked to the account. Policies for privately owned vehicles vary by region and consent settings, so both “every video is always uploaded anonymously” and “no video ever leaves the car” are misleading.

The learning loop is roughly:

  1. Engineers define a failure scenario, such as a yellow light in glare, an unusual work zone or a motorcycle cutting in.
  2. Vehicles select candidate clips and associated vehicle state matching the condition.
  3. Servers label, filter and organize training and evaluation sets.
  4. Data centers retrain models and run simulation, closed-loop and vehicle regression tests.
  5. A passing version is distributed over OTA and real-world performance is observed again.

Tesla's published simulation view of its self-driving evaluation infrastructure

<Official Tesla autonomy evaluation infrastructure image 1.1>

In Evaluation Infrastructure, Tesla describes placing anonymized characteristic clips into large test suites, generating realistic graphics and sensor data and running hardware-in-the-loop evaluation. This data engine is Tesla's most important asset. The decisive advantage is not camera count alone, but how quickly failures become targeted data, new training and regression tests that prevent old problems from returning.

Who pays for mobile data?

Tesla vehicles have cellular modems and use local carriers. Owners do not activate a separate smartphone-style data plan for FSD learning traffic. According to Tesla Connectivity, Standard Connectivity is included for eight years from new delivery and provides core maps, navigation and access to OTA. Important safety updates continue over the vehicle's cellular connection.

In the United States, Premium Connectivity is $9.99 per month or $99 per year and adds cellular live-traffic visualization, satellite maps, streaming and browser features. FSD works without Premium Connectivity. Tesla states this directly in its FAQ. Pricing and included periods vary by country.

Data function Included or operated by Tesla Optional owner payment
On-device FSD inference Runs on vehicle hardware Unrelated to Premium
Important safety updates Delivered by cellular No separate mobile plan
General OTA Accessible to all vehicles; Wi-Fi commonly recommended for large downloads Owner's Wi-Fi environment
Selected learning data Sent according to consent, policy and event conditions No per-upload charge
Satellite maps, live traffic and streaming Limited without Premium Premium Connectivity

How does OTA replace the model?

OTA for a car is more than an app update. It is deployment to a safety-critical embedded system. The vehicle downloads signed packages, verifies integrity and installs firmware, neural networks, maps or control software only when installation conditions are satisfied. Tesla's FSD support page says even subscription activation requires completion of an OTA update.

It would be speculation to claim every update distributes a full model over cellular. Tesla does not publicly detail package sizes, delta-update mechanisms or partition layout. What is confirmed is that all vehicles have access to OTA and feature availability varies by hardware, software, region and model year. Download speed is less difficult than safe recovery, power-loss tolerance, signature verification, hardware branching and regression testing.

The AI software and inference compute page from Tesla's Q1 2026 investor update

<Tesla Q1 2026 published AI software material 2.1>

Tesla's Q1 2026 investor update says FSD v14.3 improved reinforcement learning for long-tail cases, enhanced its vision encoder for low-visibility scenes, and rewrote the AI compiler and runtime to reduce inference latency by up to 20%. It demonstrates vertical integration across chips, compilers, runtime, vehicles and training infrastructure. “Up to 20%,” however, is an internal company comparison, not an independently reproduced road-test result.

An objective assessment of Tesla FSD's technical lead

There are strong reasons to rate the technology highly:

  • A very large compatible fleet that doubles as a learning and evaluation sensor network
  • Time-aware multi-camera integration, occupancy modeling and planning in one evolving AI stack
  • Vertical integration of inference chips, compiler, runtime, vehicle control, OTA and training compute
  • Consumer Level 2 assistance across an unusually broad range of city streets and highways
  • A fast data engine from failure discovery through retraining, regression evaluation and redeployment

That does not prove Tesla has the world's most advanced fully autonomous system. Consumer FSD remains supervised Level 2. The NHTSA PE25012 opening document lists complaints and crashes involving red-light entries, movement into opposing lanes and incorrect lane selection. An open investigation is not a final defect finding, but it shows regulators are examining whether drivers received enough warning and time to respond to unexpected behavior.

Tesla's FSD Safety Report reports lower collision rates with FSD engaged. Its aggregation is not a randomized comparison with identical roads, vehicle ages, driver populations, interventions and crash definitions. Selection bias matters because drivers may activate FSD more often on suitable roads. Waymo, by contrast, operates driverless Level 4 within bounded areas and publishes geographically matched human benchmarks using peer-reviewed methods. “Broad supervised Level 2” and “bounded driverless Level 4” optimize different problems and should not be reduced to one league table.

Tesla's technology stack is advanced, but product naming and the actual responsibility level must remain separate. Current FSD must not be treated as full autonomy, and company safety statistics should be read beside independent evidence.

Does faster-than-human reaction guarantee safety?

Computers do not tire, can watch several directions and can issue control commands quickly. That potential is real. Safety, however, is not one average reaction-time number. What matters is how quickly a system detects camera degradation from glare, fog or dirt; what it does when lights, signs and human directions conflict; and how early it communicates an unexpected intention to the supervising driver.

IIHS emphasizes that Level 2 cannot replace the driver and that over-trust can delay intervention. It also finds limited independent evidence that partial automation, by itself, reduces crashes beyond separate collision-avoidance features. Tesla's fleet learning is therefore both a technical strength and an ethical burden: an incomplete product is monitored by ordinary customers on public roads.

Where Hyundai and Kia need to push harder

Hyundai is not a sensor laggard. Its HDA owner documentation describes fusion of a forward camera, front and corner radar and navigation for highway lane centering, speed and distance control and assisted lane changes. Radar is a defensible engineering choice for distance measurement and adverse conditions. Yet HDA2 is mainly restricted to supported controlled-access roads and does not expose one consumer product across the variety of urban intersections and unstructured scenes that Tesla attempts.

The comparison is not about sensor count.

Dimension Tesla FSD (Supervised) Hyundai HDA2
Automation level Supervised Level 2 Supervised Level 2
Main operating scope Broad city, road and highway scenarios Supported controlled-access highway sections
Sensor strategy Camera-centric, configuration varies Camera, radar and navigation fusion
Strength Fleet data, integrated AI, OTA iteration Conservative domain, sensor redundancy, production quality
Public challenge Independent safety data, over-trust, visibility and intersection failures Urban expansion, software iteration speed, data loop

Hyundai and Kia should not simply copy a camera-only design. They need consent-based failure-data collection, common software across vehicle lines, large-scale simulation and regression evaluation, faster OTA deployment and safety metrics that outsiders can reproduce.

Tesla FSD deserves a high technical rating. It changed the industry's reference point by connecting chips, cars and data centers into a product that keeps learning after sale. But advanced is not the same as finished. Red-light and low-visibility failures remain, and the driver remains responsible. Hyundai and Kia already have strong sensor and manufacturing foundations. The next challenge is to add a faster software and data loop—and transparent safety validation—to compete for trustworthy autonomy across a much wider road domain.

Tesla FSD Technical Analysis 1 - How It Drives Without Internet

Series · Tesla FSD Technical Analysis

Article series · Completed

Episode 1 · Tesla FSD Technical Analysis 1 - How It Drives Without Internet

Tesla FSD Technical Analysis 1 - How It Drives Without Internet

This series checks Tesla FSD's architecture and limitations against public sources. The first question is simple: Can FSD keep driving in a tunnel with no cellular signal? Yes. More precisely, FSD is not “serverless” in the web-development sense. It is an on-device inference system. Training happens in data centers, while perception, trajectory planning, steering and acceleration or braking commands are computed by the vehicle's AI computer.

Tesla sells the product as Full Self-Driving (Supervised). The current consumer feature is an SAE Level 2 driver-assistance system that requires continuous supervision and immediate intervention, not autonomous driverless operation.

Is FSD serverless: driving in the car, training on servers

Calling it serverless is only half right. It does not stream every driving frame to a server and wait for an answer. Network round-trip delay, dead zones and carrier failures would become safety failures. Tesla owner manuals say the FSD computer processes front, rear and side camera inputs through neural networks and makes decisions to guide the vehicle toward its destination. Tesla's official Connectivity FAQ is even clearer: a Premium Connectivity subscription does not affect how FSD works.

The system is easier to understand as three layers.

Location Role When internet is unavailable
Vehicle Camera capture, 3D scene representation, planning, steering/braking/acceleration commands and driver monitoring Core FSD inference can continue
Tesla data centers Large-scale training, auto-labeling, simulation, regression testing and model creation No immediate effect on driving
Vehicle-cloud link OTA, maps and routing, remote app services, selected telemetry and learning clips New software and online services may be limited

Tesla's published FSD AI inference computer circuit board and chips

<Official Tesla FSD chip hardware image 1.1>

The image comes from Tesla AI & Robotics. Tesla says it designs inference chips around performance per watt, redundancy and deterministic operation, not just peak throughput. In a car, whether an answer always arrives within its deadline matters more than an impressive average benchmark.

Is the model stored as files in the car?

Yes, but it is not best understood as one giant desktop-style model file. Vehicle firmware bundles neural-network weights, execution graphs, AI-accelerator-compiled artifacts, pre- and post-processing code and safety control logic. Tesla has not disclosed the exact size or parameter count of the current model. Claims that it is a particular number of gigabytes or billions of parameters are therefore estimates, not confirmed facts.

Tesla's AI page still cites 48 networks, 70,000 GPU-hours for a full build and 1,000 output tensors per timestep. Those figures described the stack when they were published. They should not be presented as the size of the latest end-to-end FSD model. Large GPU clusters train the system; vehicles receive only the compiled, optimized inference artifacts they need.

The stage and presentation screen from Tesla's official AI Day 2022 video

<Tesla AI Day 2022 official video frame 1.2>

In the official Tesla AI Day 2022 video, the company described combining time-series video from multiple cameras into one vector space and predicting occupancy. The goal goes beyond labeling objects in a single frame: the system needs a three-dimensional, time-aware representation of what is free, what is blocked and what is moving.

What inputs does FSD use besides cameras?

FSD's central perception strategy is camera-based Tesla Vision, but that does not mean the car moves from pixels alone. Public manuals and vehicle architecture point to a wider input set:

  • Front, rear and side exterior cameras
  • The cabin camera for driver attention
  • Vehicle speed, wheel rotation, steering angle, accelerator and brake state
  • IMU acceleration, rotation and vehicle attitude
  • GPS, navigation route and road information
  • Ultrasonic sensors or radar, depending on hardware generation and market

Tesla's camera and sensor documentation also identifies ultrasonic sensors and radar on vehicles that are so equipped. The core environmental perception of recent consumer FSD is camera-centric, while exact sensor configurations vary by model, year and region.

Navigation is an intent input that provides the destination and broad route. Cameras and vehicle state are real-time inputs that determine whether the next movement is safe. Even when GPS drifts by several meters, lane centering has to be solved in a camera-derived local coordinate system.

Did Tesla train every actuator into the model parameters?

The claim that a neural network simply memorized every steering and brake characteristic is too strong. Neural networks play a growing role from perception through trajectory prediction, but vehicles still contain low-level steering, braking and drive controllers, operating limits, diagnostics and independent safeguards. A useful mental model is a hierarchy: the high-level system requests a trajectory or control target, while lower-level controllers execute it using current speed, steering angle, friction and body response.

Vehicle differences are not absorbed by a single universal parameter set. Wheelbase, steering ratio, mass, tires and brake response require configuration and calibration. A common control interface lets multiple vehicles share a high-level FSD software family. That is the practical meaning of drivetrain standardization.

Tesla's published view of low-latency vehicle code and hardware integration

<Official Tesla vehicle code foundations image 2.1>

In its Code Foundations description, Tesla identifies throughput, latency, correctness and determinism as core metrics. It describes high-frequency sensor capture and compute pipelining across multiple system-on-chips without starving central memory or safety-critical code.

This also answers the question of being faster than a human. Human hazard recognition and muscular response cannot be compared directly with one neural-network inference time. FSD's advantage is continuous, fatigue-free observation and repeated decisions. Yet fast inference is only useful when the scene is understood correctly. A fast misunderstanding is still a fast mistake.

Why older Teslas are supported, and where support ends

Tesla gained a major head start by platformizing camera positions, vehicle communications, electronic steering and braking, and OTA deployment. It could update and test a fleet, not just one model. That does not mean every older Tesla runs current FSD at identical quality.

Tesla's AI Computer installation page says some owners who bought FSD with Computer 2.0 or 2.5 are eligible for a Hardware 3 computer replacement, and some early cameras must also be replaced. Subscription customers do not receive the same complimentary upgrade terms. Tesla explicitly says availability varies with model, year, hardware, software and region. Its Q1 2026 investor material also limited the new in-car Self-Driving app to AI4 vehicles.

Broad support rests on four engineering choices:

  1. Electronic steering, braking and acceleration were abstracted behind common software interfaces.
  2. Camera placement and vehicle networking were platformized early.
  3. The AI computer was designed to be physically replaceable on some older vehicles.
  4. OTA allows repeated improvement of the same vehicle.

But camera resolution, compute, memory and placement differences prevent one unchanged model from serving every generation. Tesla's advantage is not “almost no parameter change.” It is the ability to operate one software family at fleet scale while managing hardware variants.

Part 1 has a clear conclusion: immediate FSD driving decisions happen inside the vehicle, while the cloud is the factory that trains and distributes new brains. Part 2 examines what vehicle data is uploaded, who pays for connectivity, how OTA connects to fleet learning, and how Tesla's technical lead looks when safety evidence is included.

404 Dev Room 30 - Taming

Series · 404 Dev Room Webtoon · Ongoing Episode 30 · 404 Dev Room 30 - Taming The trainer in the AI coding room has changed. <...