Live data can be captured in several ways 1. with Wireshark 2. with SDK tools 3. with the capturing functionalities in CeptonViewer (which uses the SDK). The captured data usually are created in .pcap format and could be replayed by CeptonViewer.

We will make data samples available for most current sensors. Please check back the SDK page for updates.

Current CeptonViewer support Windows 10/11, Ubuntu 18/20. Special build for other Linux distributions can be provided upon request. We are working on supporting MacOS (M1 or later) in the near future.

On Windows, the update is automatic. To manually check for updates, go to menu Help->Check For Updates.

For now, please download the latest release package (.deb) and install it over the old one. In the near future Cepton will provide pps:// servers for <code>apt upgrade</code> style updates.

Cepton Viewer currently only support .pcap (tcpdump format). You can always convert your .pcapng file to .pcap file using wireshark or similar tools

By default Cepton lidars are plug and play. There is no need to configure your local networking except for making sure the firewall is disabled for CeptonViewer or your application using SDK.

Pedestrian Classification

Imagine you are drawing a picture of a baseball game. You start drawing a batter hitting a ball. With the dynamic swing you draw, it’s very convincing to see that he has just hit a home run. Great job!! This is the most amazing drawing that I’ve ever seen befo…. 


 <img src="/images/instance_rendering/baseball_hit_man.png" width=25% height: auto/>


Wait a second. The background is missing. I can't see where he is. This home run should be in a huge baseball stadium with thousands of people. Draw them now. Of course, you should draw them with the same drawing style as the rest.

Are you reluctant to draw the audience individually? Okay, then let’s learn the "instanced rendering technique." To get ready, please forget about the baseball game. Today's focus is on the GPU rendering technique on the screen.

Instanced rendering is a technique in computer graphics
to efficiently draw tons of objects.
We can even draw hundreds or thousands of objects by reusing a single instance.
This article goes over basic notions of the instanced rendering in OpenGL
and shares some example use cases that I developed in Cepton. 
I provide sample programs at the end of each section. 

## OpenGL Basics 
Let's consider the simplest possible shape that GPU can process, a triangle. We know that a triangle consists of three points
and each vertex is represented by a 3D vector of float values.
To draw a triangle, we pass
the float array data to GPU.
However, since GPU is fully detached from CPU, we need to specify every single thing for the simple data sending. But OpenGL APIs help us to write those specifications.
The following is the essential summary of how we use OpenGL APIs to send three vertices data.
``` cpp
// Location in GPU to store the vertices data
#define VERTEX_LOCATION 0
// Generate a buffer on GPU to store data
glGenBuffers(1, &VBO);
// Specify the property of the buffer
glBindBuffer(GL_ARRAY_BUFFER, VBO);
// Vertices data
float vertices[9] = {-0.2f, -0.3f, 0.0f,
 0.2f, -0.3f, 0.0f,
 0.0f, 0.3f, 0.0f };
// Specify the pointer to the data array and its size 
glBufferData(GL_ARRAY_BUFFER,
 9*sizeof(float),
 vertices,
 GL_STATIC_DRAW);
// Tell GPU how to read the data
// I.e., the array is supposed to be interpreted as a series of 3D vectors
glVertexAttribPointer(VERTEX_LOCATION,
 3,
 GL_FLOAT, // Value type in GPU 
 GL_FALSE, 
 (GLsizei)3*sizeof(float), // Size of a 3D vector 
 NULL); // Where to start reading
// Enable the data location in GPU before drawing
glEnableVertexAttribArray(VERTEX_LOCATION);
```
Now the vertices data are ready.
Let's tell GPU to draw them on the screen,
or more formally, let's execute a "draw call."
In our example, as we want to draw a triangle,
we use GL_TRIANGLES mode
so that GPU regards the series of vectors as the vertices of a triangle, and nicely fills the inside of the triangle.

``` cpp
glDrawArrays(GL_TRIANGLES, 0, 3);
```

Here is what GPU draws.


 <img src="/images/instance_rendering/basic.png" width=60% height: auto/>


[Sample Program 1](/src/instance_rendering/basic.zip)

## Cost of Draw Calls 
Now we know how to draw a triangle on the screen.
Given that any shape can be represented by a bunch of triangles (i.e. "mesh" representation),
we can practically draw anything.
However, it's worth noting here that draw calls cannot be executed unlimitedly to ensure a reasonable refresh rate.

As a quick experiment, let's see what happens if we replace the single draw call with 1000,000 drawcalls.
``` cpp
// glDrawArrays(GL_TRIANGLES, 0, 3);
for (int i=0; i< 1000000; i++) glDrawArrays(GL_TRIANGLES, 0, 3);
```
You would find a significant delay until the triangle shows up
and have a hard time manipulating the window (below).



 <img src="/images/instance_rendering/FPS_lag.png" width=60% height: auto/>


## Instanced Rendering
OpenGL has various tricks to reduce draw call executions, one of which is 
the instanced rendering.
Suppose we want to draw 1000 objects, the instanced rendering technique encloses all the instructions into a single draw call.

Again, let's use a triangle as an example.
As we have already learned how to send the shape information of a triangle to GPU, let's go over how to specify the position information of 1000 triangle objects. 

In computer graphics, positions information is typically represented by a 4x4 transformation matrix (in the so-called "homogeneous coordinate.")

``` cpp
// Location in GPU to store the vertices data
#define MATRICES_LOCATION 1
// Number of instances
#define NUM_INSTANCES 1000

// Generate a buffer to store transformation matrices
glGenBuffers(1, &(MBO));
// Specify the property of the buffer
glBindBuffer(GL_ARRAY_BUFFER, MBO);
// Matrices data
float mat[NUM_INSTANCES * 16];
// Positions of triangles 
for (int matrix_id=0; matrix_id < NUM_INSTANCES; matrix_id++) {
 // Spiral arrangement of triangles
 float pos_x = 0.002f * matrix_id * cos(40*M_PI*matrix_id / NUM_INSTANCES);
 float pos_y = 0.002f * matrix_id * sin(40*M_PI*matrix_id / NUM_INSTANCES);
 float scale = 0.0004f * matrix_id;
 int i = 16 * matrix_id;
 mat[i+0] = scale; mat[i+4] = 0.0f; mat[i+8] = 0.0f; mat[i+12] = pos_x; 
 mat[i+1] = 0.0f; mat[i+5] = scale; mat[i+9] = 0.0f; mat[i+13] = pos_y; 
 mat[i+2] = 0.0f; mat[i+6] = 0.0f; mat[i+10] = scale; mat[i+14] = 0.0f; 
 mat[i+3] = 0.0f; mat[i+7] = 0.0f; mat[i+11] = 0.0f; mat[i+15] = 1.0f; 
}
// Specify the pointer to the array and its length
glBufferData(GL_ARRAY_BUFFER, NUM_INSTANCES * 16 * sizeof(float),
 mat, GL_DYNAMIC_DRAW);
// Tell GPU how to read the data
// The array is supposed to be interpreted as a series of 4x4 matrices 
for (unsigned int i = 0; i < 4; i++) {
 glEnableVertexAttribArray(MATRICES_LOCATION + i);
 glVertexAttribPointer(MATRICES_LOCATION + i, 4, GL_FLOAT, GL_FALSE,
 16 * sizeof(float),
 (const GLvoid *)(sizeof(GLfloat) * i * 4));
 glVertexAttribDivisor(MATRICES_LOCATION + i, 1);
}
```
Now GPU knows 1000 position information as well as the triangle shape information.
I always wish that GPU could automatically place the triangle using those matrices but there is another layer of instruction for GPU.
We need to specify how to use those matrices with the "shader program."
Although the shader programming itself is the very essence of GPU rendering, this article won’t dig into it for simplicity.
Rather let's regard it as an extra code to add more instructions on the usage of transformation matrices.
The following C-like code is the shader program.
``` glsl
layout (location = 0) in vec3 Vertex;
layout (location = 1) in mat4 Matrix;
void main()
{
 gl_Position = Matrix*vec4(Vertex.x, Vertex.y, Vertex.z, 1.0);
};
```
This shader program specifies how to process each transformation matrix and each vertex, i.e. taking the matrix product. Notice that the `Vertex` is placed at the location 0 (= `VERTEX_LOCATION`) and the `Matrix`
is in the location 1 ( = `MATRIX_LOCATION`). This shows that GPU properly receives the data at the locations that we specified.

Now that GPU finally knows everything, let's call a draw call. 
For instanced rendering, we need to use the following draw call API to draw multiple instances.
But this is counted as a single draw call, and therefore efficient.

``` cpp
glDrawArraysInstanced(GL_TRIANGLES, 0, 3, (GLsizei)NUM_INSTANCES);
```
Here is the result of instanced rendering.


 <img src="/images/instance_rendering/instanced.png" width=60% height: auto/>


Confirm that there is no delay in rendering.

[Sample Program 2](/src/instance_rendering/instanced.zip)
## Application Examples at Cepton 
Here, I share some application examples in Cepton.
In recent years, Cepton's development has been dedicated to
the object detection using the data captured by the Cepton's Lidar sensor. In my internship, I introduced the instanced rendering technique to visualize the results of object detection. 
Since the instanced rendering technique is easily extended to more complicated shapes, we used it to visualize objects commonly found during driving, such as cars and trees.

![Instances](/images/instance_rendering/instances.png)

The results were quite successful.
Although they contain a lot of triangles, (car: ~1000 triangles, tree: ~30000 triangles), they can be visualized in real-time with the instanced rendering technique (see the demo below).

![Car Demo](/images/instance_rendering/car_demo.gif)

![Tree Demo](/images/instance_rendering/tree_demo.gif)

Thank you for reading so far, and I hope this article helps your development as well.

Finally, here I would like to thank the Cepton Perception team members for the great internship experience.
I appreciated how they allowed me to try various approaches while being always willing to help me.
Without their help, I wouldn't have even hit on instanced rendering.

Cepton Viewer

Download Cepton Viewer

Documentations

Related pages

Blogs

FAQs

Join Our Team