CAN Bus: How It Works, Pros and Cons, and Fast Local Processing Tutorial

gowebing

A Control Area Network (CAN) bus is a serial communication protocol that allows devices to exchange data in a reliable and efficient way. It is widely used in vehicles, working like a nervous system to connect ECUs in the vehicle.

CAN bus was originally designed for automotive applications by Bosch in the 1980s. It is a multi-master, multi-slave, half-duplex, and fault-tolerant protocol that fits well with the requirements of automotive applications. It is simple, low-cost, and reliable and can be used in harsh environments. The CAN bus provides one point of entry for all the ECUs in the vehicle, which makes it easy to connect and diagnose.

CAN bus data can provide valuable insights into the performance and status of the connected devices. However, collecting and processing CAN bus data can be challenging due to the high data rate, low bandwidth, and variable network conditions.

One possible solution to overcome these challenges is to use MQTT, enabling timely data transmission from cars to the cloud even with weak network conditions. EMQX is an open-source MQTT broker that can help you build a reliable and scalable MQTT infrastructure for collecting CAN bus data.

A Brief History of CAN Bus

The Controller Area Network (CAN) bus, developed by Bosch, a German multinational engineering and technology company, originated in the early 1980s. Its primary purpose was to establish an effective communication system for automotive applications, specifically to decrease the complexity of wiring harnesses in vehicles.

In 1986, Bosch introduced their initial CAN protocol, which quickly gained momentum among automakers due to its reliability and robustness. By 1993, it became an international standard under ISO-11898. To summarize the evolution of the protocol:

Apart from automotive applications, other industries have embraced this versatile network protocol over time. Today, it is used in industrial automation systems (CANopen) and marine electronics (NMEA 2000). Its widespread adoption is mainly attributed to its ability to operate reliably even under harsh conditions while maintaining low-cost implementation requirements.

How Does a CAN Bus Work?

The CAN bus is a decentralized communication protocol. Its decentralized approach makes it ideal for applications in automotive and industrial systems where reliability and real-time performance are essential.

In a CAN network, all nodes are connected via twisted-pair wiring or optical fiber cables. Each node has its own microcontroller responsible for processing incoming messages and sending outgoing ones. Data is broadcasted by a node on the shared bus, allowing all other nodes to receive it. The primary stages of the communication process are:

This combination of features allows CAN buses to maintain high levels of efficiency while ensuring reliable communication between different components in complex systems like vehicles or factory automation equipment.

Message Structure in the CAN Protocol

The message structure in a CAN bus system is crucial for efficient communication between devices. The protocol uses a data frame format that consists of several fields, including an identifier, control field, data field, and error detection mechanism.

Types of CAN

Here are the three main types of CAN:

1. Low-Speed CAN

Low-speed CAN, also known as fault-tolerant or ISO 11898-3, operates at speeds up to 125 kbps. It is designed for less critical systems like body control modules, door locks, window controls, etc., where data transmission speed isn't vital. Its key feature is the ability to continue functioning even when one wire in the bus fails.

2. High-Speed CAN

High-Speed CAN, or ISO 11898-2, can reach speeds up to 1 Mbps. This type of network is suitable for more time-sensitive applications such as engine management systems and electronic braking systems due to its faster data transfer rates compared to low-speed counterparts. However, it lacks fault tolerance capabilities found in low-speed networks.

3. CAN FD (Flexible Data Rate)

CAN FD, introduced by Bosch in 2012, is an extension of high-speed networks with increased data rates—up to 5 Mbps—while maintaining backward compatibility with existing high-speed devices. The primary advantage of this technology lies in its ability to transmit larger payloads more efficiently than traditional CAN, making it ideal for modern vehicles with increasingly complex electronic systems.

CAN Bus: Advantages and Challenges

What Are the Main Advantages of CAN Bus?

The CAN bus data can provide valuable insights into the performance, health, and behavior of a vehicle. Collecting CAN bus data to the cloud is a powerful way to leverage the potential of vehicle data through big data analysis. By applying machine learning, artificial intelligence, or other analytical tools to the collected data from a large number of vehicles, vehicle manufacturers can gain valuable insights and leverage them to optimize vehicle performance.

In the AI era, data is the most valuable property. By collecting data from cars to the cloud and then distributing it to all kinds of data infrastructure like databases and data lakes, users can leverage the data for nearly all kinds of applications.

What Are the Challenges of Real-Time Data Collection?

Collecting CAN bus data locally on the vehicle is pretty mature. However, it can be challenging to collect and process the CAN bus data and transfer the insight to the cloud in real time due to the high data rate, low bandwidth, and variable network conditions. Thus, it is impractical to transfer all the CAN bus data to the cloud for processing. Instead, one can collect and process the CAN bus data locally on the edge side to reduce the data volume and transfer the insight to the cloud in real time.

We'll need at least two components to build such a solution:

1. Edge computing engine: An edge computing engine can collect only the needed CAN bus signals, flexibly process them, and trigger MQTT transfer actions in real time. LF Edge eKuiper is an open-source edge computing engine that can help you process and analyze CAN bus data in real time.

2. MQTT broker in the cloud: An MQTT broker can help transfer the processed CAN bus data to the cloud in real time. EMQX is an open-source MQTT broker that can help you build a reliable and scalable MQTT infrastructure for collecting CAN bus data.

Next, we will illustrate the overall solution combining EMQX and eKuiper.

Addressing the Challenges of CAN Bus Local Processing With eKuiper

eKuiper is an open-source edge computing engine that can help you process and analyze CAN bus data in real time. eKuiper is designed for stream processing on edge, suitable for the real-time processing of the typical streaming data generated by the CAN bus. eKuiper can address these challenges:

Tutorial: Local Processing for CAN BUS Data With eKuiper

Step 1: Connect to CAN Bus

eKuiper uses the CAN source plugin to connect to the CAN bus and receive CAN frames. It supports two modes to connect to the CAN bus, as shown in the diagram below.

  1. Connect to CAN bus directly by socketCan. SocketCAN uses the Berkeley socket API and the Linux network stack and implements the CAN device drivers as network interfaces. Once the CAN bus is connected to the Linux system, users can get the CAN network interface. In eKuiper, users can create a CAN stream by specifying the CAN network interface and the DBC file path. Any rules can then be applied to the CAN stream to process the CAN bus data.
  2. Connect to the CAN bus through a gateway by TCP/UDP. The gateway can be a CAN adapter that combines multiple CAN frames into a packet and is sent out in batch by TCP or UDP. Notice that the packet format sent by the gateway is not standardized. So, we may need to modify the plugin to adapt to it. In eKuiper, users can create a CAN stream by specifying the TCP/UDP address and the DBC file path. Any rules can then be applied to the CAN stream to process the CAN bus data.

Step 2: Decode CAN Bus Data

CAN bus data is in binary form and organized as a frame. The CAN frame is composed of several fields. Various CAN protocols include CAN 2.0A, CAN2.0B, and CANFD. The CAN frame format is slightly different for different protocols. The CAN frame format for CAN 2.0A is shown in the figure below.

Among them, two fields are important for us to decode the CAN bus data:

1. The ID field: The ID field is used to identify the CAN frame. It is 11-bit for CAN 2.0A protocol and 29-bit for CAN 2.0B and CANFD.

2. The Data field: The Data field is the payload that is used to carry the actual data. It is 0-8 bytes for CAN 2.0A and CAN 2.0B 0-64 bytes for CANFD.

In the payload, the data is organized as a series of signals. The signal is a named data item with a specific length and a specific position in the payload. DBC file is a text file that contains information for decoding raw CAN bus data to 'physical values.' It defines the signal name, length, position, and the conversion formula to convert the raw data to physical values.

In eKuiper, users can specify the DBC path to use when parsing the CAN bus data. It is pretty flexible and secure to configure the DBC in eKuiper.

After configuring the eKuiper CAN source, users can create a stream to receive the CAN bus data with physical and meaningful signals. For example, the CAN payload 0x0000000000000000 can be parsed to the following signals:

  
{
  "signal1": 0,
  "signal2": 0,
  "signal3": 0,
  "signal4": 0,
  "signal5": 0,
  "signal6": 0,
  "signal7": 0,
  "signal8": 0
}



Next, users can leverage the powerful eKuiper stream processing capabilities to flexibly process the parsed data just like receiving from MQTT.

Step 3: Process CAN Bus Data

After getting the parsed data, we can do a lot of things with it by eKuiper. In order to reduce the bandwidth to transfer data, we can pick the interested signals only. For example, we can pick the signals signal1 and signal2 only.

  
{
  "signal1": 0,
  "signal2": 0
}

The eKuiper SQL to do this is simple:

  
SELECT signal1, signal2 FROM canStream

Because CAN frame size is limited, there is a good chance that the needed signals spread around multiple CAN frames. In this case, we can flexibly composite the signals from different CAN frames to construct a complete message for applications with various algorithms according to your needs. Check the data merging example for more details. Here, we use signal1 as the main property to pick the data.

  
SELECT signal1, latest(signal2) as signal2 FROM canStream WHERE isNull(signal1) = false

Another example of processing is to collect the data only when some event happens. This can also significantly reduce the bandwidth. For example, we can collect the data only when the signal1 is higher than 100.

  
SELECT signal1, signal2 FROM canStream WHERE signal1 > 100

Moreover, these processing rules are flexible and can be changed on the fly. Don't worry if you cannot identify the needed signals at the beginning. You can change the rules to adapt to the requirement changes with hot reload.

The most mature usage is to achieve flexible data collection. Besides that, eKuiper can be used in other scenarios like:

Using MQTT to Collect CAN Bus Data

Using an MQTT broker like EMQX for collecting CAN bus data can offer several benefits, such as:

Besides these benefits, EMQX provides more features, and together with eKuiper, it can help users save bandwidth, reduce latency, and improve reliability when transferring CAN bus data.

Save Bandwidth

To transfer CAN bus data over MQTT, we usually need to transfer through weak network conditions with limited bandwidth. In this case, we need to reduce the data size as much as possible.

In eKuiper sink, we can use the format option to specify the data format. The default format is JSON. We can change it to protobuf to serialize the data into binary format to reduce the data size significantly. Additionally, we can use the compress option to compress the data by gzip or other compression methods. This way, we can significantly reduce the data size compared to the original JSON data. Especially when sending the data in batch, the data size can be reduced by 90% or more in one of our test cases.

Real-Time Data

Some of the data is time-sensitive for cloud applications. For example, the data to diagnose a vehicle accident is critical. In this case, we need to reduce the latency as much as possible. In the eKuiper rule, we can use the MQTT sink to send the data to EMQX.

To save bandwidth in the real-time scenario, we can set the serialization format and compression method as mentioned above in the eKuiper MQTT sink. On the EMQX side, it provides a rule engine which supports decompressing and deserializing the data. Without coding, the data can be consumed by the cloud application in real-time.

Batch Data in File

For data that is not time-sensitive, we can save the data in a file or local DB and send it to the cloud in batch. It can achieve a higher compression rate to save even more bandwidth. In the eKuiper rule, we can use a file sink to save and compress data locally. It supports configuring the file rolling policy. For example, we can configure the file rolling policy to roll the file every ten minutes. This way, we can save the data in a file in batch. EMQX is developing a new feature to support transferring the file. Once completed, the saved file can be transferred by MQTT. Currently, users can use other tools to transfer the file to the cloud.

Conclusion

In this blog, we have introduced how to collect, process, and transfer CAN bus data from vehicles to the cloud with eKuiper and EMQX. In the next blog post, we will go into more detail about each step.

 

 

 

 

Top