"Onion Routing" in the Lightning Network and how it works

Author: LORENZO

Computers in a network communicate with each other according to protocols. Here, "protocol" refers to a system of rules specifying how messages should be transmitted and interpreted. The payment message transmission part of the lightning network protocol is described by BOLT#4, also known as "Onion Routing Protocol (Onion Routing Protocol)".

Onion Routing is a technology that predated the Lightning Network by 25 years. It is also used in Tor, which is where the name "Tor" ("The Onion Router") comes from. The Lightning Network uses a slightly modified version called "origin-based onion routing," abbreviated "SPHINX." In this article, we're going to talk about how onion routing works.

Why use onion routing?

Many different communication protocols exist in the world, but since the Lightning Network is a payment network, it makes sense to choose a protocol that reveals as little information as possible about the payment being forwarded.

If the Lightning Network used the same protocol as the Internet, every intermediary would know who the sender of the payment was, who the recipient was, and who the other intermediaries along the path were. Onion routing is a good choice because its characteristics guarantee intermediate nodes:

• Only know your previous node (who sent you a message) and the next node (where you want to forward the message).
• does not know the length of the entire path;
• Not knowing where you are in the path.

Overview of onion routing

Let's use a parcel as an analogy to explain how onion routing works.

Suppose Alice wants to pay Dina. First, Alice needs to find a feasible path for her payment:

Alice→Bob→Chan→Dina

Then, she constructs an "onion". She starts with Dina (from the end of the path). She puts a secret message (payment content) in a package sent to Dina, and locks it with a key known only to her and Dina. Now, she puts this package in another package to be sent to Chan, and locks the package to Chan with a key known only to her and Chan. Right and so on.

Alice sends the final onion (package) to the first intermediary on the path, Bob. Bob unlocks his package with his own key, and sees that the next package is for Chan. So he forwarded the package to Chan. The same goes for Chan. After unpacking the package, he forwarded the package inside to Dina. Finally, Dina opened her own package and found the payment message inside.

In onion routing, middlemen like Bob and Chan do not know the content of the message to Dina, nor the length of the entire payment path. The only thing they know is who forwarded the package to them, and who will receive it next. This guarantees the privacy of the message and the confidentiality of the path. Each intermediary can only touch the layer of messages specially made for TA.

In the Lightning Network's source-based onion routing, the sender chooses the payment path and constructs a complete onion for that path, which can be viewed as a privacy hole (Translator's Note: The receiver's network location must be exposed to the sender ). Other routing schemes, such as "blind routing" (Chinese translation), solve this problem by obfuscating part of the payment path to the sender. However, in this article, we focus exclusively on SPHINX.

Assemble the onion

Now, let's take a look at the specification of onion routing. At the beginning, we need to define these things:

• The sender is the "initial node" (Alice);
• The receiver is the "final node" (Dina);
• Every intermediate node on the payment path is a "hop" (Bob and Chan);
• The communication information between each hop is called "hop load".

Construct jump load

Once Alice has chosen a payment path, she gets the information for each payment channel from the gossip protocol to create the payload for each hop, essentially telling each hop how to create the HTLC for the payment being forwarded ( hash timelock contract).

In order to establish a proper HTLC, each hop needs to:

• The amount that needs to be forwarded;
• secret value paid;
• The ID of the payment channel that continues to send onions;
• The length of the time lock.

Most of this data comes from "channel update" messages, which contain information about routing fees, event requests, and payment channel IDs. The total amount that needs to be forwarded is the sum of the paid amount plus the fee charged for each subsequent hop; while the secret value of the payment is calculated by Dina and embedded in the payment invoice (informed by the onion message to each one hop).

Alice starts with the final node Dina. She includes forwarding amount, time lock duration value, payment secret value and payment amount in the package. Note that she does not need to add the channel ID, because Dina is the final node and does not need to forward the payment to others.

At first glance, it seems redundant to provide the forwarding amount, because this amount is the same as the payment amount. However, multipath payment will send the payment amount through multiple paths, and then the two values will not match.

In Chan's payload, Alice adds Chan's and Dina's channel IDs. She also added forwarding amounts and timelock values. Finally, Alice creates a payload for Bob. Chan charges 100 Satoshi for the payment through the channel between herself and Dina, so Alice needs to tell Bob that the forwarded amount is the payment plus the handling fee. According to Chan's channel update message, the timelock value has also been increased by 20 (in blocks). Finally, Alice also considers Bob's handling fee and time lock requirements, and gives him an HTLC with a time lock length of 700040 and a value of 100200 Satoshi.

Shared secret value and key generation

Next, Alice prepares the onion by generating a shared secret for each hop (including the final node). This shared secret value can be generated by Alice and the target hop respectively by multiplying her own private key with the other party's public key.

A shared secret value is necessary for onion routing, allowing Alice and each hop to derive the same key. Alice then uses those keys to obfuscate each layer of the onion, and that hop uses the keys to unobfuscate.

To protect Alice's privacy, she creates a one-time session key for an onion, rather than using her own node public key, to derive the shared secret value. She uses this session key for the first hop, and then, for each subsequent hop, Alice deterministically randomizes the key by multiplying the latest key by a blinding factor. These are used to create a shared secret value key, which we call "ephemeral keys".

Bob, Chan, and Dina, all need to get the same secret value as Alice, so they need to know the ephemeral key to use in their session. Alice only puts the first key in the onion to save message size. Each hop computes the next ephemeral key and embeds it in the onion for the next node. Each hop can use its own public key and shared secret value to calculate the blinding factor used by Alice to determine the next ephemeral key.

As mentioned earlier, the shared secret value will be used to generate some keys, and Alice and the corresponding jump can use these keys to perform some operations on the onion. Let's take a look at what each key does.

Rho key

The Rho key is used by Alice to encrypt a layer of onion; this will obfuscate the contents of the payload so that it cannot be deciphered by outsiders. Only the owner of the rho key can decrypt the payload. That's what the node that receives the onion has to do: use the shared secret value with Alice to derive the rho key, then decrypt the onion and read the contents.

Mukey

Alice uses the mu key to create a checksum for each payload. She also passes the checksum to the hop that receives the onion. This hop, in turn, uses the mu key to generate a checksum of the received payload, checking that it matches the one given by Alice. This is to check the integrity of the payload, verifying that it has not been tampered with.

Pad key

This key is used only by Alice to generate random "garbage" data. These data are also part of the onion, and it has nothing to do with the length of the payment path, how many hops the onion has passed, and it keeps the onion always the same size, even if some of its contents are irrelevant. This is how onion routing hides the path length, in effect protecting the privacy of the sender and receiver.

A key

This key is also used to check the integrity of the data contained within the onion, but only if an error is returned. Yes, it's called "um" because that's "mu" written backwards. In the case of a payment error, the jump that finds the error will use the um key to create a checksum, and when the previous node receives the error report, it will also use the um key to verify the integrity of the message.

Encapsulating the onion layer

The final onion wrap looks like this:

Alice now has the payload for each hop, and the shared secret value for each hop. Let's see how Alice transforms this information into the final onion. She starts with the final node and works her way back step by step.

She first creates an empty field with a length of 1300 bytes, which is the total length of all onion payloads. She then uses the pad key to create a random string of 1300 bytes long, which is garbage that is useless for any hop. This step is done to ensure that each layer of the onion looks the same, so you can neither see the total length of the path (how many hops), nor who is the sender and who is the receiver.

She then creates a checksum of the payload that needs to be used and puts it at the end of the payload. In the message to the final node, the checksum is all 0 to inform Dina that she is the final recipient of this onion. After adding the checksum to the end of the payload, Alice puts the payload (and the checksum) at the beginning of the garbage, and deletes the part of the entire message that exceeds 1300 bytes, so that the entire message length is 1300 bytes .

Then, Alice uses the rho key to create a random byte string, and uses an exclusive-or (XOR) operation on the onion payload obtained in the previous step to obtain the obfuscated payload. The original text of the payload can be obtained by using the XOR operation of this random byte string on the obfuscated text (Translator's Note: In other words, XOR here is the algorithm of symmetric encryption, and the random byte string is the key). The XOR operation compares the onion payload against the random byte string (generated by the rho key) bit by bit, outputting a 1 only if one of the data bits is a 1; this results in an obfuscated payload. The clever thing about the XOR operation is that as long as you get the correct random byte string and the obfuscated payload, you only need to run the XOR operation with the two again to get the obfuscated payload.

Since the nodes receiving the onion can derive the same rho key, they can generate the same random byte string as Alice. This is how each node along the way can untangle the confusion and read the content.

After preparing the confusion onion for one hop, Alice repeats the same steps for the next node. The key difference is that once Dina's onions are done, she no longer needs to generate garbage. She just appends the obfuscated onion from the previous step after the useful payload and checksum, and trims off anything over 1300 bytes.

Finally, Alice takes the final obfuscated onion and adds a checksum so Bob can verify the integrity of the onion. Alice then adds the session public key so Bob can use this public key to compute the shared secret value. Finally, she also adds a byte representing the version, telling other nodes how to interpret the data in it. For the version described by BOLT #4, the version byte shall be 0.

forward onion

To send this onion package, the sender creates an update_add_htlc message with the following fields:

• Channel ID: The specific channel to which this message relates.
• ID: The identifier of this HTLC.
• Amount: The value of this HTLC.
• Payment Hash: Created by the recipient of the payment.
• Expiration time: This HTLC will expire after a certain block.
• Onion Parcel: The onion created for this payment, which is what was mentioned above.
• Additional data: used to specify additional data.

After preparing the message, Alice sends the message to Bob. After receiving the message, Bob can start decoding his own onion. He first obtains the session key from the onion wrapper, and uses it to derive the value of the shared secret with Alice.

Armed with the shared secret value, Bob generates the mu key to verify the checksum of the payload embedded in the onion package. If the payload has not been tampered with, the checksums should match.

To prevent other nodes in the path from knowing how long the path is, Bob adds a 1300-byte field filled with zeros to the onion packet. Bob then generates a 2600 byte long random byte string from the rho key. Bob uses this random byte string to XOR the zero-filled onion payload.

Remember how I told you about confusing onion loads? Use the obfuscated onion payload as input, and run the "XOR" operation with the same byte string to get the onion payload before obfuscation. Because Alice and Bob use the same shared secret value, generating the same rho key, Bob can unobfuscate. This has the added bonus that it turns the 1300-byte long pad characters into random bytes.

Bob's unobfuscated payload includes the payload data for his hop along with a fingerprint. Bob saves this fingerprint so he can add it to the onion package he sends to Chan. After Bob separates his own payload from the onion message, he converts the onion packet back to its original size of 1300 bytes and randomizes his session key as Alice did. Finally, Bob adds the version bytes, the session key, and the fingerprint he intends to put in the onion payload, and forwards the onion packet to Chan via an update_add_htlc message.

This process will continue until the message is sent to the final node, Dina. When Dina receives the update_add_htlc message, she can enter the hash value of the secret value generated by herself, which means that this HTLC is intended for her. So Dina just needs to check the fingerprints, unravel the onion messages, and reveal her own payload.

Troubleshooting

We're talking about a success story, a case where everything went according to plan, but if something goes wrong along the way, you have to send a message back all the way to notify all the nodes that something went wrong. This process is similar to regular onion routing. Spotting a faulty node requires deriving the um key from the shared secret value, using it to generate a random byte string, and using an XOR operation to obfuscate the returned onion parcel.

A node that finds an error will send a message back to the previous node in the payment path. Each hop uses the um key and ammag key to do the same operation until the sender receives the package. Finally, the sender uses the ammag key and um key to unobfuscate and verify the package.

Errors can be caused by onion packages, nodes or channels. If you use the Lightning Network a lot, you may have encountered errors such as "channel unavailable" or "not enough fees".