# Introduction
In the development of StudioViz, a simulator created by Cepton’s in-house software engineers, we aim to implement a feature for detecting collisions between objects. This feature, known as Collision Detection, addresses the computational problem of identifying when two or more spatial objects intersect. It is a critical component in computer graphics, simulation, and various other applications where accurate spatial interactions between objects are necessary.

In the realm of games and simulation engines, collision detection plays a crucial role in ensuring realistic interactions between objects and their environment. This process involves calculating whether and when objects intersect, providing essential feedback for simulating real-world physics and behaviors. By accurately detecting collisions, developers can create environments where objects respond naturally to impacts, enhancing the realism and immersion of the simulation. While this doesn't involve direct obstacle avoidance, it establishes a foundational framework for evaluating collision scenarios during development and testing. This capability is particularly valuable for refining perception algorithms, ultimately contributing to the broader vision of safe and autonomous transportation by ensuring that simulated models behave as expected in various scenarios.

To achieve collision detection, various methods such as bounding volume hierarchies, spatial partitioning, and proximity queries may be employed. These techniques aim to minimize computational overhead while maintaining high accuracy in detecting collisions. The implementation of Collision Detection feature in StudioViz enhances the simulator's capabilities, enabling more realistic and interactive simulations for users. In this blog post, we will cover the algorithm behind collision detection and demonstrate various scenarios that simulate real-world car collisions using Pygame, a comprehensive set of Python modules designed for writing video games.


# Collision Detection Algorithm Explanation
When it comes to collision detection for cars, several methods are available, each with distinct advantages and disadvantages. These methods include Axis-Aligned Bounding Box (AABB), Oriented Bounding Box (OBB), and Separating Axis Theorem (SAT). Here is a detailed examination of each method:

| Methods | Pros | Cons |
|:-------------:|:--------------|:--------------|
| AABB  | Computationally inexpensive<br>Easy implementation  | Inaccurate for objects that are not axis-aligned   |
| OBB  | Better accuracy and versatility    | More complex implementation   |
| SAT  | Even higher accuracy and versatility | Most complex implementation |

We have ultimately selected the Separating Axis Theorem (SAT) for car collision detection due to its superior accuracy, despite its complex implementation. When dealing with lidar systems, it is crucial to ensure that collisions are detected with the highest precision. SAT provides the necessary level of accuracy, making it the most suitable choice for our application.

SAT is quite simple in concept. Basically, if you are able to draw at least one straight line between two objects, then those two objects are not colliding. If you are unable to, then they are colliding.


<p align="left">
  <img src="/images/2d_collision_detection/Figure1.png"  width=50% height: auto/>
</p>
<small>Figure 1: In this diagram, the two squares are not colliding, and you are able to draw a straight line through them. </small>


<p align="left">
  <img src="/images/2d_collision_detection/Figure2.png"  width=50% height: auto/>
</p>
<small>Figure 2: In this diagram, the two squares are colliding, and it is impossible to draw a straight line that separates the two. </small>


We refer to each straight line as an “axis”. But unlike the usual x-axis and y-axis that you may be used to, any straight line is considered an axis. SAT axes are not just limited to horizontal and vertical axes.


<p align="left">
  <img src="/images/2d_collision_detection/Figure3.png"  width=50% height: auto/>
</p>
<small>Figure 3: All four of these lines are considered “axes”. </small>


We quickly ran into an issue when writing code for this algorithm. If any straight line is considered an axis, then there are any infinite number of axes to test. This makes it impossible to compute collision detection through sheer brute force.


<p align="left">
  <img src="/images/2d_collision_detection/Figure4.png"  width=50% height: auto/>
</p>
<small>Figure 4: Since any straight line is considered an “axis”, there are technically an infinite number of axes. </small>


The beauty of SAT comes from its systematic methodology of determining which axes to test. Generally speaking, the SAT operates by projecting the vertices of the shapes onto potential separating axes, which are typically the normal vectors to the edges of the shapes. For each axis, the algorithm checks whether the projections of the two shapes overlap. If there exists at least one axis where the projections do not overlap, it indicates a separating axis, and the shapes are not colliding. Conversely, if the projections overlap on all tested axes, the shapes are intersecting. This method ensures precise collision detection by accounting for the actual geometry and orientation of the objects involved.


<p align="left">
  <img src="/images/2d_collision_detection/Figure5.png"  width=50% height: auto/>
</p>
<small>Figure 5: Example showing colliding objects. </small>


This example demonstrates the collision of convex objects in a 2D coordinate system. The green dashed line represents a potential separating axis, while the red and orange lines depict the projections of Polygons A and B onto this axis. As the projections overlap along the green line, a collision occurs between the two polygons. 

# Visualizing in Pygame

## Created an object class
We created a self-defined class called ‘SimplifiedObjects’ (class) to represent a two-dimensional rectangular object with various functionalities for manipulating its position, rotation, and checking for collisions with other objects of the same type.

### Attributes
This class stores essential attributes such as the vertices of the rectangle, its center, dimensions (width and length), the direction it is facing, the angle of rotation, and a flag indicating if it has collided with another object.

### Methods
Getters and Setters: Allow us for easy access and modification on attributes, such as angle and coordinate positions.

Movements: Allow us to intuitively manipulate the moving directions.

Collision Detection: Apply the SAT to the class to check collisions.

Helper Methods: Calculate distances between points, updating the object's center, direction, and angle, and rotating its vertices.

## Show_objects
We created another function called ‘show_objects’ to render a collection of objects onto a Pygame screen. Each object is represented by an image, which is scaled to fit within the screen dimensions while maintaining its aspect ratio. The function also handles rotation of the images based on the angle provided by each object. It detects and highlights collisions by overlaying a semi-transparent red bounding box on the colliding objects. 

# Demo

## Basic Movements
In this part, we will demonstrate the basic movement of a car within the Pygame. This example will showcase the fundamental functionalities to users for simulating vehicular motion.



<div style="display: flex; justify-content: center; gap: 10px;">
  <img src="/images/2d_collision_detection/Move_Forward.gif" style="width: 25%; height: auto;" />
  <img src="/images/2d_collision_detection/Move_Backward.gif" style="width: 25%; height: auto;" />
  <img src="/images/2d_collision_detection/Coordinate_movements.gif" style="width: 25%; height: auto;" />
  <img src="/images/2d_collision_detection/Rotate.gif" style="width: 25%; height: auto;" />
</div>



## Scenarios
In this section, we aim to showcase various scenarios demonstrating car movements within Pygame, specifically focusing on collision and non-collision events. These scenarios will highlight the ability to accurately visualize and handle different collision dynamics, providing valuable insights into the behavior of vehicles in diverse situations.



<div style="display: flex; flex-wrap: wrap; justify-content: center; gap: 10px;">
  <img src="/images/2d_collision_detection/Head-On_Collision.gif" style="width: 30%; height: auto;" />
  <img src="/images/2d_collision_detection/Head-On_Miss.gif" style="width: 30%; height: auto;" />
  <img src="/images/2d_collision_detection/Backing_Up_Collision.gif" style="width: 30%; height: auto;" />
  <img src="/images/2d_collision_detection/Turning_Collision.gif" style="width: 30%; height: auto;" />
  <img src="/images/2d_collision_detection/Turning_Miss.gif" style="width: 30%; height: auto;" />
  <img src="/images/2d_collision_detection/Parallel_Parking.gif" style="width: 30%; height: auto;" />
</div>



# Future Improvements

The next step in enhancing StudioViz involves transitioning from 2D to 3D visualization, incorporating more complex and realistic elements such as car models, trees, and pedestrians. This will provide a more immersive and accurate simulation of real-world scenarios. This comes with the challenge of creating collision detection for irregular shapes - making sure that objects know whether they are coming into contact with the ground, buildings, and trees. Eventually, we would also like to use machine learning to predict and prevent collisions based on historical data and real-time inputs.

# References

"Separating Axis Theorem," *Wikipedia*,  https://en.wikipedia.org/wiki/Hyperplane_separation_theorem.

Kah Shiu Chong, "Collision Detection Using the Separating Axis Theorem", envatotuts+, last modified August.6th 2012, https://code.tutsplus.com/collision-detection-using-the-separating-axis-theorem--gamedev-169t

OpenAI, ChatGPT, response to an question in Separating Axis Theorem example , June.19th 2024.

Collision Detection


# Overview
Cepton’s lidar sensors send out two kinds of packet on a regular basis: Point Data Packet and INFO Data Packet. Please refer [Data Format](https://developer.cepton.com/doc/data_format) for detailed structure information. Chapter 2 in this document outlines the implementation of Precision Time Protocol (PTP) in Cepton’s products to illustrate the generation of critical PTP related values reported in those two kinds of packets. These values are mainly used for displaying real-time system information, or run-time self-monitoring and diagnostic. Chapter 3 gives an example of how to use those reported values to calculate universal PTP timestamp for each point. Chapter 2 is for your better understanding, but you can skip this Chapter and directly refer to chapter 3 for usage only. 

# Reported Value Generation 
##  Clock adjustment
Each slave clock maintains a local counter calculating the number of microseconds (us) since the sensor last booted up. When connection with PTP master clock becomes active, the clock adjustments shall perform synchronization and syntonization. Therefore, the universal PTP timestamp calculation is generally composed of three parts: 
<center>

$ptp\_timestamp = local\_counter + synchronization\_correction + syntonization\_correction$

</center>

Synchronization eliminates the offset between slave’s counter and master’s clock.  Syntonization matches the frequency of local clock to that of the Grandmaster Clock in the domain, which is measured by the slave's counter increases every drift interval, it is one nanosecond slower than the master's clock. Thus, the formula becomes more concrete: 
<center>

$ptp\_timestamp = local\_counter + master\_slave\_offset + \frac{local\_counter\_increase}{drift\_interval}$

</center>

Clock adjustments are carried out through a sequence of PTP timing messages exchange between upstream PTP Instance and local slave. The basic message exchange pattern is illustrated in IEEE Std 1588-2019 Figure 16 as shown below : 
<p align="center">
  <img width="460" src="/images/ptp/message_exchange.png">
</p> 

(a) The Master PTP Instance sends a Sync message to the Slave PTP Instance and notes the time `t1` at which it was sent. The timestamp `t1` can be embedded in the `Sync` message or in a `Follow_Up` message. 

(b) The Slave PTP Instance receives the `Sync` message and notes the time of reception `t2`.   

(c) The Slave PTP Instance sends a `Delay_Req` message to the Master PTP Instance and notes the time `t3` at which it was sent. 

(d) The Master PTP Instance receives the `Delay_Req` message and notes the time of reception `t4`. The Master PTP Instance conveys to the Slave PTP Instance the timestamp `t4` by embedding it in a `Delay_Resp` message or in a `Delay_Resp_Follow_Up` message.

The message exchange pattern illustrated in 2.1 is repeated contiguously to maintain latest `master_slave_offset`  and  `drift_interval`, which will be explained in 2.2 and 2.3 independently.

## Update `master_slave_offset` 
Say master sends a message to the slave at PTP time `m1`, after a communication link propagation delay `d`, the slave receives message at its local counter time `c1`, then the offset currently equals to: $Δ1 = c1 - (m1+d)$. The link propagation delay is required for updating the offset. Assuming symmetric link, that link delay is a mean of t<sub>ms</sub>
 and t<sub>sm</sub>. However, (t1, t4) and (t2, t3) follows different time metrics, therefore, the delay is calculated by:   
<center>

$d = \frac{{(t_2 - t_1) + (t_4 - t_3)}}{2} = \frac{{(t_2 - t_3) + (t_4 - t_1)}}{2}$

</center>

Initially, it’s set to zero.   

Since link delay can only be updated every time a complete group of $<t1, t2, t3, t4>$ is obtained, the newest offset is calculated based on the last group of $<t1, t2, t3, t4>$.

## Update `drift_interval`
Differentiate from offset, which depends only on the last group of message exchange, The drift_interval that will be reported is an accumulated value, which is updated every time a new pair of (t1, t2) is recorded. As mentioned in 2.1, the slave's counter increases every drift interval, it becomes one nanosecond slower or faster than the master's clock. According to this, what is being maintained is drift_interval, but what is being added to timestamp calculation is: $\frac{local\_counter\_increase}{drift\_interval}$.
The `drift_interval`  represents a time range of counter in nanosecond unit and is updated as follows: 
<center>

$drift\_interval = \frac{counter\_range}{master\_range - counter\_range}$

</center>

where:

<center>

$master\_range = current\_master\_arrival\_timestamp - last\_master\_arrival\_timestamp$

$counter\_range = current\_counter\_ns - last\_counter\_ns$

</center>

The drift_interval can be a positive value or negative value: 

 - when drift_interval is positive, it means counter timestamp is increasing slower than master clock; 

 - when drift_interval is negative, it means counter timestamp is increasing faster than master clock; 

 - when drift_interval is zero, which means counter timestamp is increasing in the same speed as master clock, syntonization_correction will also be zero.
  
## Report PTP relevant values 
Recall the PTP timestamp calculation formula:  

<center>

$ptp\_timestamp = local\_counter + master\_slave\_offset + \frac{local\_counter\_increase}{drift\_interval}$

</center>

We have explained the generation of master_slave_offset in 2.2 and the generation of drift_interval in 2.3. The local_counter is the slave’s internal non-decreasing counter timestamp (currently using emac system time, in us unit). The value counts the time interval between the moment of power-up (emac power-up) to the moment of INFO packet sent-out. 

Cepton’s lidar sensors send out two kinds of packet on a regular basis: Point Data Packet and INFO Data Packet.  

In INFO packet header, there are three fields containing PTP related information, which are filled in the local_counter value, master_slave_offset value, and drift_interval value correspondingly: 
| Field Name              | Unit   | 
|-------------------------|--------|
| time_since_powerup      | us     |
| time_offset_from_master | us     | 
| time_drift_correction   | per ns | 

In Point Data Packet’s payload, each point has a relative_timestamp field that holds the relative counter value from the time since last point in the packet. For the very first point, it is time since the packet header's reference time.     

# PTP Usage Example
The Cepton SDK usage is almost all about using Callback APIs, please refer [SDK Guide](https://developer.cepton.com/doc/sdk_guide) for more information. Here, only point’s PTP timestamp calculation relevant part will be explained. 

The master_slave_offset and drift_interval is only reported in INFO packet, therefore, to calculate following points’ timestamp, a local copy of previous INFO packet information is needed, the callback looks like this: 

```c
int64_t last_info_offset   = 0;
int64_t last_info_drift    = 0;  
int64_t last_info_counter  = 0;  

void infoCallback(CeptonSensorHandle handle, 
            const struct CeptonSensor *info, void *user_data) {
  last_info_counter = info->power_up_timestamp; 
  last_info_offset  = info->time_sync_offset; 
  last_info_drift   = info->time_sync_drift;  
} 
```
Since the calculation is based on previous INFO packet, the local counter increase in the formula becomes the time difference between each point’s counter and the counter timestamp when the previous INFO packet is sent out. Therefore, the power_up_timestamp in INFO packet must also be recorded. 

Each point’s counter timestamp calculation is a litter bit confusing. Recall that each point has a relative_timestamp field that holds the relative counter value from the time since last point in the packet. For the very first point, it is time since the packet header's reference time.  Here is how you can get each point’s counter value: 

```c
int64_t prev_point_counter = 0;  

void pointCallback(CeptonSensorHandle handle,
              int64_t start_timestamp, size_t n_points,
              size_t stride, const uint8_t *points,
              void *user_data){
    prev_point_counter = start_timestamp; 
    for (size_t i = 0; i < n_points; i++) { 
        struct CeptonPoint point = *(struct CeptonPoint const*)(points + i * stride); 
        int64_t point_counter = prev_point_counter + (int64_t)point.relative_timestamp;  
        prev_point_counter = point_counter; 
    } 
}
```

Based on above obtained value and formula, each point’s timestamp can be calculated as: 
```c
#include <stdio.h>
int64_t drift_correction = (last_info_drift==0) ? 0 :
 (point_counter - last_info_counter)*1000 / last_info_drift; 
printf("calculated point PTP timestamp(ns): %lld \n",  
  (last_info_counter - last_info_offset)*1000 + drift_correction  
); 
```
Pay attention to the unit conversion. 

Finnally, a complete sample code for your reference: 

```c
#include <stdio.h>
#include <stdlib.h>

#include "cepton_sdk2.h"

void check_api_error(int err, char const *api) {
  if (err != CEPTON_SUCCESS) {
    printf("API Error for %s: %s\n", api, CeptonGetErrorCodeName(err));
    exit(1);
  }
}

void sensorErrorCallback(CeptonSensorHandle handle, int error_code,
                         const char *error_msg, const void *error_data,
                         size_t error_data_size) {
  printf("Got error: %s\n", error_msg);
}

int n_info = 0;

//(per ns)
int64_t last_info_drift    = 0; 
//(1us)
int64_t last_info_counter  = 0; 
int64_t last_info_offset   = 0;
int64_t prev_point_counter = 0;

void infoCallback(CeptonSensorHandle handle, 
            const struct CeptonSensor *info, void *user_data) {
    printf("Got %d info:", ++n_info);
    printf("ts_info_counter: %lld ts_offset: %lld ts_drift: %lld\n",  
          info->power_up_timestamp, 
          info->time_sync_offset,
          info->time_sync_drift
    );
    last_info_counter = info->power_up_timestamp;
    last_info_offset  = info->time_sync_offset;
    last_info_drift = info->time_sync_drift;
  
}

void pointCallback(CeptonSensorHandle handle,
              int64_t start_timestamp, size_t n_points,
              size_t stride, const uint8_t *points,
              void *user_data){
    prev_point_counter = start_timestamp;
    for (size_t i = 0; i < n_points; i++) {
        struct CeptonPoint point = *(struct CeptonPoint const*)(points + i * stride);
        int64_t point_counter = prev_point_counter + (int64_t)point.relative_timestamp; 
        if (i == 0 || i == n_points-1) {
            printf("the raw counter(us) of point %d is: %lld\n", (int)i, point_counter);
            int64_t drift_correction = (last_info_drift==0) ? 0 : (point_counter - last_info_counter)*1000 / last_info_drift;
            printf("calculated point PTP timestamp(ns): %lld \n", 
                (last_info_counter - last_info_offset)*1000 + drift_correction 
            );//temporary - offset
            
        }
        prev_point_counter = point_counter;
    }
}

int main() {
  int ret;

  // Initialize
  ret = CeptonInitialize(CEPTON_API_VERSION, sensorErrorCallback);
  check_api_error(ret, "CeptonInitialize");

  ret = CeptonEnableLegacyTranslation();
  check_api_error(ret, "EnableLegacyTranslation");

  // Start networking listener thread
  ret = CeptonStartNetworking();
  check_api_error(ret, "CeptonStartNetworking");

  // Listen for points
  ret = CeptonListenPoints(pointCallback, 0);
  check_api_error(ret, "CeptonListenPoints");

  // Listen for info
  ret = CeptonListenSensorInfo(infoCallback, 0);
  check_api_error(ret, "CeptonListenSensorInfo");
  
  // Sleep
  while (n_info < 10)
    ;

  // Deinitialize
  ret = CeptonDeinitialize();
  check_api_error(ret, "CeptonDeinitialize");
  return 0;
}
```
This example prints the counter, offset and drift in each INFO packet, and the raw counter and calculated timestamp of the first and last points in each Point Data packet, feel free to modify the output to best fulfil your requirement. 

PTP Implementation and Point Timestamp Calculation


## Introduction
Cepton's perception team is actively developing a system to classify point cloud objects acquired from their LiDAR sensor.
Neural networks are widely known as the state-of-the-art approach to tackle the task of classification,
and Cepton LiDAR's high-resolution point cloud can be used with them, although efficient and accurate featurization has to be completed first.

In my internship, I integrated a voxel featurization into their system and tested its capability.
In this post, I will go over how point cloud data are featurized as voxel data and how the neural network architecture is tailored for voxel featurization.
In the end, I briefly report that my voxel featurization showed high efficiency and accuracy for the object classification task.


## Featurization

Data collected from our LiDAR sensors takes the shape of a point cloud, with each point having position information in the form of $x, y, z$ coordinates.
However, these unstructured points can’t easily be used in a neural network, so they must be converted to some structured form first, i.e. the input data has to be featurized.
There are many different ways of doing this, with the one I chose being a 3D voxel grid.


I start with normalizing the points to the origin.
I compute the minimum value for each dimension across the entire point cloud and then translate all points by subtracting these minimum values, aligning the point cloud with the origin.
I use cubical voxels of size $0.1m$, each of which is set to be 1.0 if it contains a point in it, otherwise 0.0.
I set the number of voxels as $24^3$ (2.4m each), to well capture both pedestrians and vehicles.

Once this was done for all of the points, it resulted in a collection of voxels that represented the point cloud.
Finally, I converted the grid into a tensor, which could then be fed into a neural network, which I will explain next. 

```python
import numpy as np
import torch

def featurize(point_cloud):
  # Find min and normalize
  minX = min(data, key=lambda x: x[0])[0]
  minY = min(data, key=lambda x: x[1])[1]
  minZ = min(data, key=lambda x: x[2])[2]
  data = [(x - minX, y - minY, z - minZ) for [x, y, z] in data]
  width = 0.1
  # Create grid and fill with zeros
  grid_size = 24
  voxelGrid = np.zeros((min_grid_size, min_grid_size, min_grid_size), dtype=float)
  # Fill in grid
  for [x, y, z] in data:
      indexX, indexY, indexZ = int(x / width), int(y / width), int(z / width)
      indexX = min(indexX, grid_size - 1)
      indexY = min(indexY, grid_size - 1)
      indexZ = min(indexZ, grid_size - 1)
      voxelGrid[indexX, indexY, indexZ] = 1

  # Convert a NumPy array to a Tensor
  voxelTensor = torch.from_numpy(voxelGrid)
  return voxelTensor
```
<div style="text-align: center;">
<img src="/images/voxel_classifier/pointcloud_pedestrian.png" width="350" height="350">
<img src="/images/voxel_classifier/voxelized_pedestrian.png" width="350" height="350">
</div>

<div style="text-align: center;">
<img src="/images/voxel_classifier/pointcloud_vehicle.png" width="350" height="350">
<img src="/images/voxel_classifier/voxelized_vehicle.png" width="350" height="350">
</div>


## Neural Network Architecture

After featurizing the point cloud data, the next step is to feed them to a convolutional neural network (CNN) to build a classifier.
The reason that this has to take place after featurization is that the point cloud data lacks a clear structure and features to learn from. However, featurizing the points as voxels gives CNN a structure to work with, which greatly benefits the learning process.

This type of neural network consists of multiple convolutional layers with max pooling and activation functions,
followed by fully connected layers and a final output layer.
For my classifier, which I implemented in PyTorch Lightning, I pretty much followed the <a href="https://cs231n.github.io/convolutional-networks/#architectures" target="_blank">standard structure</a>
: I had three convolutional layers, each followed by a max pool and ReLU, then two fully connected layers using a linear transformation, with the first one followed by ReLU, and finally a softmax and output layer.
Because the data was in three dimensions, I used the three-dimensional versions of the convolution and max pool functions.

Throughout this process, the dimensions go from $24 \rightarrow 22 \rightarrow 11 \rightarrow 9 \rightarrow 4 \rightarrow 2 \rightarrow 1$ while the number of filters goes from $1 \rightarrow 16 \rightarrow 32 \rightarrow 64 \rightarrow 32 \rightarrow 3$ and the kernel size stayed constant at $3 \times 3 \times 3$. See the figure below for the details.  
<div style="text-align: center;">
<img src="/images/voxel_classifier/nn.png">

Note: this figure isn't completely accurate because the visualization tool doesn't support three-dimensional convolutions, but it's a fair approximation.
</div>

## Results and Revisions
The classifier achieved good accuracy, with > 99% on pedestrians and > 90% on vehicles.
The speed was also reasonably fast, although not the fastest out of all the classifiers available.
| True/Predicted | Unknown | Pedestrian | Vehicle |
| -------------- | ------- | ---------- | ------- |
| Unknown        | 9.69%   | 0.0%       | 0.0%    |
| Pedestrian     | 0.0%    | 65.39%     | 0.06%   |
| Vehicle        | 0.0%    | 2.71%      | 22.15%  |

| Inference Batch | Object Count | Time (ms) |
| --------------- | ------------ | --------- |
| Single          | 1            | 0.83      |
| Small Batch     | 10           | 7.28      |
| Large Batch     | 100          | 89.47     |

### Further Extension
As an extension, I experimented with dropping the final convolutional layer and its associated max pool and ReLU, while reducing the initial dimensions of the grid to a $10 \times10 \times 10$. This resulted in the number of filters topping out at $32$, and the dimensions going from $10 \rightarrow 8 \rightarrow 4 \rightarrow 2 \rightarrow 1$. These changes were effective, with the classifier's speed improving significantly ($\times 35$ faster) while accuracy remained the same.

| Inference Batch | Object Count | Time (ms) |
| --------------- | ------------ | --------- |
| Single          | 1            | 0.26      |
| Small Batch     | 10           | 0.81      |
| Large Batch     | 100          | 2.59      |


<div style="text-align: center;">
<img src="/images/voxel_classifier/nn2.png">
</div>

## Conclusion
In this post, I went over the voxel featurization and the CNN architectures to featurize and classify point cloud data. 
The classifier I implemented was able to achieve high accuracy and speed. In the future, the accuracy could be improved with greater amounts of training data, and the speed could likely be increased with further refinements to the neural net architecture.

## References
<a href="https://cs231n.github.io/convolutional-networks/#architectures" target="_blank">Convolutional Neural Network Structure from Stanford</a>


Point Cloud Object Classification with Voxel Featurization

<p>Firmware should only be updated when it is necessary to fix a bug or access a new feature. Customer should consult with a Cepton field support engineer before undertaking an update. The tool to update firmware and latest firmware image will be provided by field support engineers upon request. Publicly available update tool and current firmware image may be provided for download in the future.</p>

<p>Currently 802.1AS is supported, with full support planned for later.</p>

<p>Yes. Full support for Python (Python 3.6 or later) and Python specific APIs are part of our SDK release.</p>

<p>Yes. Javascript bindings to SDK is currently in pre-release beta, available upon request.</p>

<p>Yes. C# bindings to SDK is currently in pre-release beta, available upon request.</p>

Developer Portal

Downloads

Documentations

Showcase

Blogs

Videos

FAQs

Join Our Team