0xicbor
A port of intel/tinycbor from C to Rust, linkable as
a drop-in libtinycbor.a.
tinycbor is Intel's CBOR (RFC 8949) implementation: roughly 6,500 lines of C, aimed at microcontrollers, allocation-free on the common path. This is that library rewritten in Rust with the C ABI preserved byte for byte, so existing callers do not notice the swap.
$ make
$ cc -Icrates/cbor-ffi/include yours.c -o yours \
target/release/libtinycbor.a -lm -lpthread -ldl
The claim, and how to check it
Any port can say it works. The check here is that upstream's own test suite (Qt/C++) is compiled and linked against the Rust library with no edits, and whatever it reports is the score.
The suite lives in tests/original/, copied verbatim from commit 9441b2ca, with SHA-256
hashes pinned at kickoff:
$ cd tests/original && sha256sum -c hashes.txt
./CMakeLists.txt: OK
./parser/tst_parser.cpp: OK
...
If a test cannot pass, it stays failing and gets a paragraph in the decision log. A 94% pass rate you can reproduce beats a 100% claim you cannot.
Status
Real output of make test and bench/run.py, not targets.
| Original suite | 4,929 / 4,929, zero failures |
| Symbol parity | 44 / 44, zero nm diff against upstream |
| ABI layout | asserted against C-dumped offsets, passing |
unsafe blocks | 80, all in the shim; cbor-core at zero |
| Differential fuzz | 13.5M execs, zero divergences, four targets |
| Tools vs upstream | exact, 4,509 documents by 20 flag combinations |
| Dependencies | zero |
| Speed vs C | 3.4x faster (mean p50 over 16 measurements) |
Every row that can apply to a port passes. The two that cannot are tst_cpp, which
#includes upstream's .c files and therefore tests C sources this port does not have.
Parsing is 2.7x faster than the C on every corpus file and printing 4.4x. It was 1.49x slower on all eight when it was a literal transliteration, and every figure since is kept in the benchmark's history table rather than quietly replaced. The one measurement the C still wins is process startup, published as prominently as the rest. See the scoreboard.
Seven bugs have been found in this port and fixed. Two by the fuzzer, five by tests written for code that nothing else reached. They are written up rather than summarised away, because the interesting question about a port is not whether it had bugs but how they were caught. One bug was found in upstream and filed: intel/tinycbor#331.
Where to start
You want to use it. Using the library has a program that compiles and its real output, then encoding and parsing.
You want to know whether to trust it. The scoreboard is every number in one place, and differential fuzzing is the honest version, including what it found.
You want to know how it was built. Why an ABI shim is the one architectural decision everything else follows from, and where the C ends is precisely what is and is not C here, because "no C" is a claim worth being able to verify.
You are porting something yourself. The cookbook is the parts that
generalise: getting to a red test loop, matching an ABI, matching printf exactly, and what
to do when the C is faster and you cannot see why.
Verified 2026-08-02, against upstream commit 9441b2ca, on Linux x86-64.
Using the library
The build produces target/release/libtinycbor.a. It exports the same 44 symbols as
upstream's archive of the same name, and the headers in crates/cbor-ffi/include/ are
upstream's, byte for byte. So if you already link tinycbor, you link this the same way and
change nothing else.
$ make
$ cc -Icrates/cbor-ffi/include yourprogram.c -o yourprogram \
target/release/libtinycbor.a -lm -lpthread -ldl
Those last three flags are the Rust standard library's own link requirements. They are the only difference from linking the C, and they are not optional.
The smallest thing that works
Encode a two-key map into a stack buffer:
#include <stdio.h>
#include <cbor.h>
int main(void)
{
uint8_t buf[64];
CborEncoder root, map;
cbor_encoder_init(&root, buf, sizeof buf, 0);
cbor_encoder_create_map(&root, &map, 2);
cbor_encode_text_stringz(&map, "id");
cbor_encode_uint(&map, 7);
cbor_encode_text_stringz(&map, "ok");
cbor_encode_boolean(&map, true);
cbor_encoder_close_container(&root, &map);
size_t len = cbor_encoder_get_buffer_size(&root, buf);
for (size_t i = 0; i < len; ++i)
printf("%02x", buf[i]);
printf("\n%zu bytes\n", len);
return 0;
}
$ cc -std=c99 -Icrates/cbor-ffi/include encode.c -o encode \
target/release/libtinycbor.a -lm -lpthread -ldl
$ ./encode
a262696407626f6bf5
9 bytes
Nine bytes for {"id": 7, "ok": true}. That is the entire pitch for CBOR, and the reason
tinycbor exists.
What you get, and what you do not
No allocation on the parse path. Not "few allocations", none. You give the encoder a
buffer and you give the parser a buffer, and the library never asks the heap for anything.
The one exception is cbor_value_dup_text_string, which is documented to malloc and hand
you the result. That is decision 20.
No errno, no global state. Every call returns a CborError. Two calls on two threads
touching two different encoders cannot interfere, because there is nothing shared to
interfere with.
No dynamic dispatch you did not ask for. The parser reads a flat buffer by default. If
your bytes arrive some other way, cbor_parser_init_reader takes a table of callbacks
instead. Both go through the same code.
Not a Rust API. This is a C library that happens to be written in Rust. There is no
serde integration, no #[derive], no Value enum. If you want an idiomatic Rust CBOR
crate, several good ones exist and this is not one of them. What this is for is replacing
the C in a codebase that already speaks C.
Reading on
- Encoding: buffers that are too small, indefinite lengths, and why the encoder keeps going after it runs out of room.
- Parsing: containers, strings that arrive in chunks, and the cursor rules.
- The command-line tools:
cbordumpandjson2cbor.
If what you actually want is to know whether to trust any of this, start at the scoreboard instead.
Encoding
An encoder writes into memory you own. cbor_encoder_init takes the buffer and its size,
and from then on the encoder never asks anyone for more room.
uint8_t buf[64];
CborEncoder root;
cbor_encoder_init(&root, buf, sizeof buf, 0);
Containers get their own encoder, which is how the library tracks how many items you still owe without allocating a stack:
CborEncoder array;
cbor_encoder_create_array(&root, &array, 3);
cbor_encode_uint(&array, 1);
cbor_encode_uint(&array, 2);
cbor_encode_uint(&array, 3);
cbor_encoder_close_container(&root, &array);
Write to the child while it is open, write to the parent after you close it. Mixing those up is the most common way to produce a document that decodes into something you did not expect, and nothing will stop you.
Running out of room is not an error you have to handle immediately
This is the part of the API most people miss, and it is genuinely good design.
When the buffer fills, the encoder does not stop. It quietly switches from writing bytes to
counting them, returns CborErrorOutOfMemory from that call and every call after it, and
keeps tracking exactly how much space the whole document would have taken.
uint8_t small[4];
CborEncoder e;
cbor_encoder_init(&e, small, sizeof small, 0);
CborError err = cbor_encode_text_stringz(&e, "far too long for four bytes");
printf("err = %d (%s)\n", err, cbor_error_string(err));
printf("extra bytes needed = %zu\n", cbor_encoder_get_extra_bytes_needed(&e));
err = -2147483648 (out of memory/need more memory)
extra bytes needed = 25
So you can encode into a deliberately tiny buffer, ignore every return value, ask at the
end how much you needed, allocate exactly that, and encode again. Two passes, no guessing,
no realloc loop. On a microcontroller that is often the difference between fitting and
not.
The bytes already written stay valid. The encoder truncates at the boundary rather than half-writing an item.
Indefinite lengths
Pass CborIndefiniteLength when you do not know the count up front. The encoder writes the
_ form and the break byte on close.
CborEncoder array;
cbor_encoder_create_array(&root, &array, CborIndefiniteLength);
cbor_encode_uint(&array, 1);
cbor_encode_uint(&array, 2);
cbor_encoder_close_container(&root, &array);
That produces 9f 01 02 ff, four bytes, against 83 01 02 03-style three for the definite
form of the same length. You pay one byte for not knowing.
Closing containers, checked and unchecked
cbor_encoder_close_container trusts you. cbor_encoder_close_container_checked verifies
you wrote exactly as many items as you promised and returns CborErrorTooFewItems if you
did not.
Use the checked one while you are developing. The unchecked one exists because on a device where you have already proven the shape, the comparison is dead weight.
Not writing to a buffer at all
cbor_encoder_init_writer takes a callback instead of a buffer. Every fragment the encoder
produces is handed to your function, tagged with a CborEncoderAppendType saying whether
it is structural CBOR, string payload, or raw passthrough. Return CborNoError to accept
it, anything else to stop the encode.
That is how you stream straight to a socket without buffering the document first. It is also the path that had no test coverage anywhere until the encoder fuzz target started driving it.
Floats, and the one that bites
cbor_encode_float and cbor_encode_double write what you give them. cbor_encode_half_float
takes a pointer to a raw 16-bit pattern you already have.
cbor_encode_float_as_half_float is the interesting one: it converts, and conversion means
rounding. It rounds to nearest, ties to even, matching what the C gets from a hardware
_Float16 cast. If you are round-tripping floats through half precision and comparing for
equality, that rounding is where your surprise is going to come from, and it is worth
reading decision 19 before you file a bug.
Next steps
- Parsing, which is the same ideas pointed the other way.
- The scoreboard if you want to know how much of the above is checked against upstream rather than asserted.
Parsing
A parser is a cursor over bytes you own. Nothing is copied, nothing is allocated, and the
CborValue you get back points into your buffer. Keep the buffer alive for as long as you
keep the cursor.
CborParser parser;
CborValue it;
CborError err = cbor_parser_init(doc, sizeof doc, 0, &parser, &it);
if (err) {
fprintf(stderr, "%s\n", cbor_error_string(err));
return 1;
}
Keep the CborParser alive too. CborValue holds a pointer back to it, and neither the
compiler nor this library will notice if you let it go out of scope first. That is a real
sharp edge inherited from the C API, and decision 21 explains
why it could not be fixed without breaking the ABI.
Walking a map
CborValue map;
cbor_value_enter_container(&it, &map);
while (!cbor_value_at_end(&map)) {
char key[16];
size_t n = sizeof key;
cbor_value_copy_text_string(&map, key, &n, &map);
if (cbor_value_is_unsigned_integer(&map)) {
uint64_t v;
cbor_value_get_uint64(&map, &v);
printf("%s = %llu\n", key, (unsigned long long)v);
} else if (cbor_value_is_boolean(&map)) {
bool v;
cbor_value_get_boolean(&map, &v);
printf("%s = %s\n", key, v ? "true" : "false");
}
cbor_value_advance(&map);
}
cbor_value_leave_container(&it, &map);
On {"id": 7, "ok": true} that prints:
id = 7
ok = true
Three rules hold this together.
A map is a flat sequence. Key, value, key, value. There is no pair abstraction. If you advance an odd number of times you are now reading values as keys and the library will not warn you.
enter and leave come in pairs. cbor_value_leave_container is what moves the outer
cursor past the whole container. Skip it and the outer cursor is still sitting on the
opening byte.
Every call can fail. The examples here drop the return values to stay short. Do not do that in code that runs.
Strings, and the length parameter that goes both ways
cbor_value_copy_text_string takes a buffer and a pointer to its size. On success the size
becomes the length written. If the buffer is too small you get CborErrorOutOfMemory and
the size becomes the length you needed, so the two-pass pattern works here as well:
size_t len = 0;
cbor_value_calculate_string_length(&value, &len);
char *s = malloc(len + 1);
cbor_value_copy_text_string(&value, s, &len, NULL);
s[len] = '\0';
cbor_value_dup_text_string does that for you and hands back a malloced pointer. It is
the only function in the library that allocates, and the memory is yours to free.
Strings can arrive in pieces. An indefinite-length string is a series of chunks with no
single contiguous form on the wire. The copy functions hide that: they walk the chunks and
join them into your buffer. cbor_value_get_text_string_chunk exposes the chunks directly
if you would rather not have a buffer big enough for the whole thing.
Skipping things you do not care about
cbor_value_advance moves past the current item, whatever it is. On a container that means
the entire subtree, however deep.
This turns out to be the hottest function in the library, and how it works is the reason this port parses faster than the C. Upstream descends recursively and fully decodes every item on the way past, then throws all of it away. This one scans for the end instead, because the only thing an advance produces is a position. It hands back to the recursive path on anything unusual, so the errors you get are still the original errors. That is decision 16, and the measurements are in the benchmark methodology.
Bytes from somewhere other than a buffer
cbor_parser_init_reader takes four callbacks instead of a pointer: can you read this many
bytes, read them, advance, and hand me a string. Use it when the document is arriving over a
transport and you do not want to assemble it first.
The rest of the API is identical afterwards. Internally the two are separate specialised copies of the parser rather than one copy testing a flag, which is decision 13.
One honest caveat: the subtree scan above is buffer-only by construction, so a reader source
takes the recursive path and is slower. It is correct, and it is checked against upstream by
tests/port/tst_reader_diff.c on every make test, but it is not the fast path.
Validation is separate
Parsing accepts anything structurally well formed. If you need more, cbor_value_validate
takes a flag matrix: shortest-form integers, sorted map keys, unique keys, valid UTF-8,
no undefined, no tags, finite floats, and a dozen more. CborValidateStrictest turns on
everything.
Run it before you trust a document from outside. It is a separate call because most callers do not need it and it is not free.
Next steps
- The command-line tools, which are the fastest way to see what a document actually contains.
- Differential fuzzing for how the above is checked against the C rather than merely believed.
The command-line tools
Two binaries, both rewritten in safe Rust rather than being C over the library. Same flags, same output, same exit codes as upstream's.
$ cargo build --release -p cbordump -p json2cbor
cbordump
Reads CBOR, prints it. Diagnostic notation by default, JSON with -j.
$ cbordump doc.cbor
{"id": 7, "ok": true}
$ cbordump -j doc.cbor
{"id":7,"ok":true}
Diagnostic notation (RFC 8949 §8) is
the one to reach for when you are debugging, because it shows you things JSON cannot say:
tags, byte strings, indefinite lengths, undefined, the difference between a float and an
integer that happens to be whole.
| flag | effect |
|---|---|
-c | diagnostic notation (default) |
-j | JSON |
-f | show text and byte string fragments separately |
-n | show overlong encodings of numbers and lengths |
-M | with -j, add metadata so the conversion back is exact |
-O | with -j, turn CBOR tags into JSON objects |
-S | with -j, stringify map keys that are not text |
-U | with -j, base64url every byte string regardless of tags |
Malformed input goes to stderr and the exit code is the CborError. Whatever was decoded
before the failure has already been printed, which is deliberate and matches upstream:
$ cbordump truncated.cbor
{"id
truncated.cbor: unexpected end of data
$ echo $?
1
json2cbor
The other direction.
$ cbordump -j doc.cbor | json2cbor - > out.cbor
- reads stdin. -M interprets the metadata cbordump -jM adds, which is what makes the
round trip lossless. Without it, JSON has no way to express a tag or a byte string, so the
trip through JSON quietly flattens them.
They are a second implementation, and that mattered
These tools do not call libtinycbor.a. They cannot: the FFI speaks in raw pointers, and a
binary using it would need unsafe outside the one crate allowed to have any. So they are a
separate parser and a separate printer built on cbor-core.
Which means nothing was checking them. Upstream's Qt suite tests the library. The fuzzers call the C ABI directly. A tool could have disagreed with upstream on every input and the board would still have read green.
Running them against upstream's own binaries over 4,509 documents and every flag
combination found four bugs, one of which accepted a document upstream rejects. The harness
is tests/port/tools_diff.sh and it runs in make test. The write-up is
decision 18.
Next steps
- Encoding and Parsing for the library behind the tools.
- Troubleshooting if
make test-toolsis skipping.
Scoreboard
Real output, updated as the port moves. Nothing here is a target.
Original suite
| Binary | Passed | Failed | Notes |
|---|---|---|---|
tst_encoder | 1,596 | 0 | |
tst_parser | 2,506 | 0 | |
tst_tojson | 827 | 0 | |
tst_c90 | 1 | 0 | header compiles under -std=c90 -pedantic |
tst_cpp | n/a | n/a | inapplicable, see decision 2 |
| Total | 4,929 | 0 | of 4,929 reachable |
Everything that can pass, passes, and nothing is stubbed any more. The last one was
_cbor_value_dup_string, which no row in the suite reaches. It has its own
differential test under tests/port/ instead, run by make test against upstream's own archive.
ABI
| Check | Result |
|---|---|
| Exported symbols | 44 / 44 |
nm diff vs upstream | empty |
| Struct sizes and offsets | asserted, passing |
| C90 header cleanliness | passing |
Budget
| Parse speed vs C | 0.37x, faster on all 8 files (2.7x) |
| Pretty-print vs C | 0.23x (4.4x), but see methodology |
unsafe blocks | 80 (all in cbor-ffi) |
| Third-party dependencies | 0 |
| Differential fuzz | 13.5M execs, zero divergences, four targets, history |
| Tools vs upstream | exact on 4,509 documents x 20 flag combinations, decision 18 |
| Decision log entries | 25 |
Reading this honestly
4,929/4,929 is every row that can apply to a port. It is not 4,931 because two rows test that upstream's C compiles as C++, which a Rust port cannot satisfy by construction.
The speed numbers under it need reading carefully in the other direction. Parsing is faster
than the C on all eight corpus files now, but it was 1.49x slower on all eight for a day,
and the history table in the methodology keeps that figure
rather than erasing it. The print number is 4.4x and most of it is upstream's vfprintf
per byte, not decoding, which the methodology says before it quotes it.
If the port ends below 100%, the failing tests stay failing and each gets a paragraph in the decision log. A reproducible 94% is worth more than a 100% that required editing the suite.
Two of those rows need a footnote about what make test actually runs. The 4,509
documents were the accumulated fuzz corpus, which is generated and not committed, so a
fresh clone checks the nine documents that are: 180 cases plus a 16-document round trip.
Point the harness at your own corpus to repeat the wide sweep:
$ tests/port/tools_diff.sh fuzz/corpus/pretty_diff/*
The fuzz total is the same shape of claim, answered by
the ledger. Comparing the tools also needs upstream's cbordump
and json2cbor binaries, which are not vendored the way the archive is; without them
make test-tools says so and skips, and the 4,929 rows above are unaffected. Every other
target here, make fuzz and make bench included, runs from a fresh clone.
Verified 2026-08-02. Reproduce with make test, make symbols, make lint, python3 bench/run.py.
Running the original suite
The tests under tests/original/ are upstream's, byte for byte. They are never edited,
never regenerated into, and their hashes are checked at ship time.
$ cd tests/original && sha256sum -c hashes.txt
./CMakeLists.txt: OK
./c90/tst_c90.c: OK
./cpp/tst_cpp.cpp: OK
./encoder/data.cpp: OK
./encoder/tst_encoder.cpp: OK
./parser/data.cpp: OK
./parser/tst_parser.cpp: OK
./tojson/tst_tojson.cpp: OK
...
How they get linked against Rust
Upstream drives its suite from CMake through a tinycbor_add_qtest() helper. Reproducing
that would mean carrying a slice of upstream's build system for no gain. The only things
the helper does that matter here are run moc and link Qt Test, which is four lines of
makefile:
$(BUILD)/tst_%: $(ORIG)/$$*/tst_%.cpp $(LIB)
@mkdir -p $(BUILD)
$(MOC) $(QT_CFLAGS) -I$(INCLUDE) $< -o $(BUILD)/tst_$*.moc
$(CXX) $(CXXFLAGS) -I$(BUILD) -I$(ORIG)/$* $< -o $@ $(LIB) $(QT_LIBS)
The test sources are inputs and nothing else. moc output goes into build/, which is why
tests/original/ still hash-verifies after a full build.
The test that cannot pass
tst_cpp.cpp opens like this:
#include "../../src/cborencoder.c"
#include "../../src/cborparser.c"
#include "../../src/cborvalidation.c"
It is not a test of the library. It is a test that upstream's C sources compile cleanly as
C++. A Rust port has no .c files for it to include, so the test is inapplicable by
construction rather than failing on behaviour.
It contributes 2 rows, which is why the reachable total is 4,929 rather than upstream's 4,931. Leaving it out of the makefile is a presentation choice; the reason is recorded so the missing rows are not mistaken for a gap.
The test that survived
tst_c90.c includes only cbor.h and compiles under -std=c90 -pedantic -Wall. It tests
the header, which this port still ships, so it remains a live constraint: the vendored
headers have to stay C90-clean.
What the runner prints
$ make test
== original suite, linked against the Rust libtinycbor.a ==
tst_encoder 1596 passed 0 failed
tst_parser 1134 passed 1372 failed
tst_tojson 2 passed 825 failed
tst_c90 1 passed 0 failed
---
TOTAL 2732 passed 2197 failed (upstream: 4929)
Per binary, not just a total, because the binaries fail for different reasons and watching one of them move while the others do not is how you tell which module you actually just finished.
Verified 2026-08-02 against upstream commit 9441b2ca.
Differential fuzzing
The original test suite proves this port satisfies the cases Intel thought to write. It cannot prove anything about the cases nobody thought of. That is what the fuzzer is for: generate arbitrary bytes, feed the identical bytes to this port and to upstream's C, and compare what comes back.
The oracle runs in another process
The obvious way to build this is to link upstream's C into the fuzz binary and call it directly. It would be faster and simpler, and it would quietly make the central claim of this port, that the shipped library contains no C, false.
So fuzz/oracle/cbor-oracle is a standalone executable built from upstream's sources,
driven as a subprocess: bytes in on stdin, the rendering on stdout, the CborError
on stderr. Nothing about it is linked into libtinycbor.a. The cost is roughly 2,000
executions per second instead of tens of thousands, which is a real price, and it is
worth paying to keep the claim checkable. This is
decision 5.
Four targets
pretty_diff compares cbor_value_to_pretty_advance. json_diff compares
cbor_value_to_json_advance, and it exists because the two share a parser and very
little else.
JSON has to refuse things diagnostic notation renders happily: a map key that is not a
string, an integer too large to survive a double, a byte string with no tag saying how
to encode it. It has its own escaping rules, its own base64 and base16, and a $cbor
metadata sidecar. Fuzzing the printer reaches none of that.
CborToJsonFlags also changes the output substantially, turning tags into objects and
byte strings into base64url and map keys into strings, so json_diff takes the bitmask
from the first byte of each input and passes the same value to both sides. A harness
that only ever sent the default would leave most of the converter dark.
validate_diff compares cbor_value_validate, which is a predicate rather than a
renderer: there is no output to diff, so the error code is the whole answer, and that
makes it the strictest of the four. Its CborValidationFlags is a 27-bit matrix where
almost every bit gates a separate check: shortest-form integers and floats, sorted
maps, unique keys, tag use, UTF-8, finite floats, unknown simple types. So the first
four bytes of each input are the bitmask and the rest is the document.
encode_diff is the odd one, and it is the one that mattered most.
The first three read CBOR. Half of tinycbor writes it, and nothing differential had ever touched that half, for a reason that looks like a good one until you look twice: an encoder takes calls, not bytes, so there is no input to hand it.
So the fuzzer's bytes become the calls. Each input is a little program: a two-byte output buffer size, then a stream of opcodes with their operands, interpreted twice, once against this port and once against the oracle, covering every encoder entry point including the container stack. What is compared is the error from every single call, the bytes each side wrote, and how much more room each says it needed.
The buffer size is the operand that matters. Upstream's encoder does not stop when it
runs out of room: it turns its union from a write pointer into a byte counter, keeps
walking the calls, and reports at the end what a big enough buffer would have taken.
That bookkeeping is most of the non-obvious code in cborencoder.c, and it is
unreachable with a buffer that always fits. So the size comes out of the input, small,
and most programs overrun on purpose.
Bit 15 of that size word picks cbor_encoder_init_writer instead: no buffer at all, a
callback per fragment. It is a separate branch through the same append() with its own
idea of running out of room, and the CborEncoderAppendType it hands the callback is ABI
surface that nothing else checks, so the callback records that argument alongside the
bytes, and refuses everything past the same size limit so the error travelling back out
through the encoder gets exercised too. That one bit was worth 40 edges.
The cost of this design is a second interpreter. The program format is specified once,
in a comment above run_encoder_program in the oracle, and implemented twice. The two
have to agree about the program before they can disagree about the encoder, which means
an early divergence is much more likely to be the two readers than the two encoders.
Thirty hand-written seed programs covering every opcode were replayed through both sides
before the fuzzer ran at all.
$ ./fuzz/run.sh # the printer, the default
$ TARGET=json_diff ./fuzz/run.sh # the converter
$ TARGET=validate_diff ./fuzz/run.sh # the validator
$ TARGET=encode_diff ./fuzz/run.sh # the encoder
Each target keeps its own corpus and its own log.
What is compared, and what deliberately is not
#![allow(unused)] fn main() { if our_err != their_err { panic!("DIVERGENCE (error code)") } if their_err == 0 && our_out != their_out { panic!("DIVERGENCE (stdout)") } }
Error codes are compared always. Output bytes are compared only when the render succeeded, and that exception is not laziness.
Upstream streams its output through a printf-style callback as it walks, so a render
that fails half way through has already emitted the first half. This port renders into
a String and writes once at the end, so a render that fails emits nothing at all. On
one real input that is 2,906 bytes against zero. The two are not comparable and never
will be; what is comparable on a failure is which error was reported, and that is
compared strictly.
That design choice is in the pretty printer's module comment and it costs nothing a caller can observe on the success path, which is the only path where the API promises specific bytes.
Run history
Stated in full, including the run that found something.
| execs | duration | result |
|---|---|---|
| 103,703 | 60 s | clean |
| 252,830 | 121 s | clean |
| 420,793 | 900 s | one divergence, see below |
| 1,507,421 | 901 s | clean, after the fix |
| 511,224 | 301 s | clean, after the string-walk change |
| 529,508 | 301 s | clean, after the subtree scan landed |
| 922,346 | 601 s | clean, after the scan learned indefinite lengths |
And json_diff, which is newer:
| execs | duration | result |
|---|---|---|
| 242,469 | 121 s | clean |
| 1,388,985 | 901 s | clean |
| 744,685 | 601 s | clean, after the CborErrorIO fix |
It reaches 1,188 edges against the printer target's 765, which is the answer to whether it was worth adding: about 55% more of the library, none of it previously fuzzed.
validate_diff:
| execs | duration | result |
|---|---|---|
| 362,801 | 301 s | clean |
| 1,677,406 | 901 s | clean |
encode_diff, which found something in its first quarter hour:
| execs | duration | result |
|---|---|---|
| 2,653 | first run | one divergence, see below |
| 2,032,920 | 1,201 s | clean, after the fix |
| 1,644,600 | 901 s | clean, with the writer callback path added |
encode_diff reaches 2,171 features on 433 edges. The edge count is lower than the
parser targets because the encoder is a smaller body of code; the feature count is the
highest of the four, which is what a target that varies both the calls and the room they
have to work in should look like. Adding the writer path took it from 393 edges to 433.
Two clean runs before a real bug is the whole argument for running it longer than the minimum. Sixty seconds of differential fuzzing is enough to claim you did it. It is not enough to find anything past the shallow water. The bug that was there took seven minutes and four hundred thousand executions to surface, and both earlier runs had already reported success.
The run after the fix went 3.6x further than the one that found the bug, on a corpus the earlier runs had grown, and stayed clean. Every later entry is a re-verification after a change to the parser, because a change that touches how bytes are walked is exactly what this is for.
What it found
The printer's missing type
A 1,220-byte input of deeply nested maps and tags. This port returned
CborErrorUnexpectedEOF (257); upstream returned CborErrorUnknownType (259).
The parser was not the problem. Running cbor_value_advance over the same bytes gave
the identical error on both sides. The difference was in value_to_pretty, which had
no arm for CborInvalidType and fell through to a catch-all that reported the input
had run out. Upstream has an arm for it:
case CborInvalidType:
err = stream(out, "invalid");
if (err)
return err;
return CborErrorUnknownType;
It prints the word invalid and reports the type, which is the more accurate claim:
the input has not necessarily run out, the item is simply not there. The case is
reachable because container_to_pretty checks for the end of a map before rendering
the key and then renders the value unconditionally, so a map that runs out between a
key and its value hands value_to_pretty a cursor whose type is Invalid.
Getting there needs about 1,024 levels of nesting, which is why no small input
reproduces it. cargo fuzz tmin could not shrink the case below 1,199 bytes, and a
brute-force search over every input up to five bytes found nothing. It is exactly the
kind of case a hand-written test suite does not contain.
Reading the same function to fix it turned up a second, quieter gap: upstream calls
copy_current_position(it, &recursed) on both error exits from the array/map case, so
a caller inspecting its cursor after a failure sees where the descent actually stopped.
This port returned the error and left the cursor on the opening bracket. Same error
code either way, and nothing would ever have caught it, but the state left behind was
wrong.
Both are fixed, and the input is now a permanent fixture under tests/port/corpus/,
replayed by make test in a couple of seconds rather than waiting on libFuzzer to
rediscover it.
Is the same arm missing anywhere else? In the library, no. cbortojson.c and
cborvalidation.c have the same case CborInvalidType: return CborErrorUnknownType;,
and this port's tojson.rs and validation.rs already had it. The pretty printer was
the one that did not. And, it later turned out, so was the second pretty printer in
tools/cbordump/, which is a separate rewrite nothing was checking. That is
decision 18.
The encoder's off-by-one
encode_diff found one at 2,653 executions, on its first run.
cbor_encode_simple_value rejected the range 24..=31 as reserved. Upstream's guard is
value >= HalfPrecisionFloat && value <= Break, and HalfPrecisionFloat is 25:
if (value >= HalfPrecisionFloat && value <= Break)
return CborErrorIllegalSimpleType;
24 is the escape byte that introduces a two-byte simple value, so upstream accepts it
and writes f8 18, which upstream's own parser then refuses, as a value under 32
written in two bytes. The encoder writes what the parser will not read.
This port now does the same, for the reason decision 17 gives about the UTF-8 bug: the claim being made here is equivalence with a specific commit, and a port that quietly fixed upstream's asymmetries would make its own differential fuzzer meaningless.
What is worth noticing is why it survived. tst_encoder has 1,596 rows and passes
every one of them. None of them asks for simple value 24. A hand-written suite tests the
values a person thought of, and 24 sits in the gap between "obviously fine" and
"obviously reserved", which is exactly the shape of thing a fuzzer walks into in the
first four seconds and a person does not write down in an afternoon.
Running it
$ ./fuzz/run.sh # 60 seconds, stops at the first divergence
$ ./fuzz/run.sh 900 # 15 minutes
$ KEEP_GOING=1 ./fuzz/run.sh # keep going past divergences, collect them all
Everything is teed to fuzz/log.txt. A divergence leaves a reproducer in
fuzz/artifacts/pretty_diff/ and exits non-zero.
Where the totals come from
Each run overwrites its target's log, so fuzz/log*.txt only ever shows the most recent
run of each. That is fine for reading the last result and useless for anyone checking a
number like "13.5M executions", which sums runs whose logs are gone.
So fuzz/history.tsv is the ledger. One row per run, with the commit it ran against:
date target seconds execs commit result
2026-08-01 pretty_diff 901 1507421 c60944c clean
2026-08-02 encode_diff 901 1644600 f8077e0 clean
run.sh appends a row when it finishes, so the file stays true without anyone
maintaining it. The published totals are that file added up:
$ awk -F'\t' 'NR>6 {e+=$4; s+=$3} END {print e, s}' fuzz/history.tsv
13457075 8054
Fourteen runs, 13,457,075 executions, 8,054 seconds, nothing outstanding. The two
divergences above are not rows in it: the runs that found them had their logs replaced by
the commits that fixed them, which is exactly the gap the ledger exists to close. Their
reproducers are in tests/port/corpus/ instead, which is the stronger evidence anyway.
Verified 2026-08-02. Re-run any of it with TARGET=<name> ./fuzz/run.sh <seconds>.
Why an ABI shim
The requirement was to run the original test suite unmodified. That single constraint decides the whole architecture, so it is worth walking through why.
The problem
Upstream's tests are Qt/C++ files that call the C API directly:
CborParser parser;
CborValue value;
cbor_parser_init(data, len, 0, &parser, &value);
QCOMPARE(cbor_value_get_type(&value), CborIntegerType);
There are three ways to make that compile and run against Rust, and two of them are wrong.
Rewrite the tests to call a Rust API. This is the tempting one, and it is the one that makes the score meaningless. A test suite you rewrote is a test suite you can make pass. Every disagreement between your port and upstream becomes a test edit rather than a bug.
Wrap the Rust in a hand-written C translation layer. Better, but it introduces a second implementation between the tests and the port, one that can absorb bugs, paper over mismatches, and quietly encode the behaviour you expected rather than the behaviour upstream had.
Hand the tests a libtinycbor.a they cannot distinguish from the real one. This is what
0xicbor does. cbor-ffi is a crate-type = ["staticlib"] exporting the same symbols with
the same signatures against the same headers. The tests are compiled exactly as upstream
compiles them and linked against a library that happens to be Rust.
What that costs
The port carries the shape of a C API at its boundary. cbor_value_get_int_checked returns
its result through an out-parameter because callers expect that:
CborError cbor_value_get_int_checked(const CborValue *value, int *result);
Inside cbor-core the same operation is what you would write if nobody were watching:
#![allow(unused)] fn main() { pub fn get_int_checked(&self) -> CborResult<i32> }
The out-parameter convention exists in exactly one place, the shim, and cbor-core never
sees an integer error code. That split is the recurring shape of this port: idiomatic Rust
in the middle, C conventions only at the edge, and one thin layer translating between them.
What it buys
Two properties become mechanically checkable rather than matters of trust:
- Layout parity: the structs are byte-identical to C's, proven by a test against numbers dumped from a C program.
- Symbol parity:
nmon this library and on upstream's produce the same sorted list, and the diff is empty.
Neither of those is an argument. They are commands you can run.
Layout parity
cbor.h declares 103 public functions. Only 44 of them are real exported symbols. The other
59 are static inline and read struct fields directly:
CBOR_INLINE_API CborType cbor_value_get_type(const CborValue *value)
{
return (CborType)value->type;
}
That function compiles into the caller, which is the Qt test binary, not into
libtinycbor.a.
No amount of correct Rust can influence what it returns. If CborValue::type is at a
different offset in the Rust struct than the C header believes, the test reads a different
byte and the failure looks like a parser bug rather than a layout bug.
So the struct layouts are not an interface this port gets to design.
Pinning the numbers
At kickoff a throwaway C program dumped every size, alignment and offset. It was run
outside the repository so no C landed in the tree, and its output was committed to
crates/cbor-ffi/abi-layout.txt:
sizeof CborEncoder = 32
alignof CborEncoder = 8
offsetof CborEncoder . data = 0
offsetof CborEncoder . end = 8
offsetof CborEncoder . remaining = 16
offsetof CborEncoder . flags = 24
sizeof CborParser = 16
sizeof CborValue = 24
offsetof CborValue . parser = 0
offsetof CborValue . source = 8
offsetof CborValue . remaining = 16
offsetof CborValue . extra = 20
offsetof CborValue . type = 22
offsetof CborValue . flags = 23
Those numbers are then asserted in a Rust test, so the tripwire trips at cargo test
rather than three hours into debugging a phantom parser failure.
The union question
All three structs contain a C union:
union {
uint8_t *ptr;
ptrdiff_t bytes_needed;
CborEncoderWriteFunction writer;
} data;
Rust has union, and using it would be the more literal translation. This port uses a
single pointer-sized word instead:
#![allow(unused)] fn main() { #[repr(transparent)] pub struct Word(pub *mut c_void); }
Every member of every one of these unions is exactly pointer-sized and pointer-aligned, so the layouts are identical, and the layout test proves it rather than assuming it.
The reason to prefer the word is the unsafe budget. Reading any field
of a Rust union is unsafe, whether or not it can actually misbehave, because the
compiler cannot know which member is live. Since the unsafe count in this port is a
published number, spending blocks on field reads that carry no real risk would inflate it
without buying any safety. Which member is live is decided by the owning struct's flags,
exactly as it is in C.
Verified 2026-08-02 on Linux x86-64.
Symbol parity
If the exported symbol list matches upstream's exactly, any program that links against tinycbor links against this instead. That is the strongest statement a drop-in replacement can make, and it is one command:
$ make symbols
symbols: 44/44, zero diff
Which runs:
$ nm -g --defined-only target/release/libtinycbor.a \
| awk '$2 ~ /^[A-TV-Z]$/ {print $3}' | grep -E '^_?cbor_' | sort -u \
| diff bench/reference/symbols-upstream.txt -
bench/reference/symbols-upstream.txt was generated the same way from upstream's own
libtinycbor.a, built at kickoff and committed as the comparison target. It is never
linked into anything.
The 44
Thirty-seven are documented API. Seven are the private-but-exported functions that
cbor.h's inline accessors call, and they are part of the ABI whether or not they are part
of the API:
_cbor_value_begin_string_iteration
_cbor_value_copy_string
_cbor_value_decode_int64_internal
_cbor_value_dup_string
_cbor_value_finish_string_iteration
_cbor_value_get_string_chunk
_cbor_value_get_string_chunk_size
Missing any one of those produces a link error in the test binary, not a runtime failure, which is a pleasant category of bug to have.
Getting it right the first time
The scaffold exported all 44 as stubs before a single line of CBOR logic existed, and the diff was empty on the first build. That ordering is deliberate: symbol parity is a property of the interface, so it costs almost nothing to establish up front and becomes a regression test for every commit afterwards. Discovering a missing symbol at hour 60 is a much worse afternoon.
Verified 2026-08-02. Reproduce with make symbols.
Where the C ends
"No C in the shipped artifact" is a claim, and claims about absence are the easiest ones to get quietly wrong. This page states exactly what is and is not C, so it can be checked rather than believed.
The library contains no C
libtinycbor.a is compiled entirely from Rust. There is no cc crate, no
bindgen, no build.rs invoking a C compiler, and nothing links against
upstream. The dependency graph of cbor-ffi is one entry long and it is cbor-core, which
itself has none.
$ cargo tree
cbor-ffi v0.1.0
└── cbor-core v0.1.0
There is no third-party CBOR crate either
Not ciborium, not serde_cbor, not minicbor, and not half. Wrapping an existing crate
is not a port. cbor-core is #![no_std] with alloc, and the IEEE 754 binary16
conversions are written out by hand, about thirty lines that would otherwise have been a
dependency.
What is C, stated plainly
The headers, in crates/cbor-ffi/include/. These are upstream's cbor.h,
cborjson.h, cborinternal_p.h, compilersupport_p.h and two generated headers, vendored
so a fresh clone builds without tinycbor checked out beside it. They are the ABI contract.
The 59 static inline accessors in cbor.h compile into whatever program includes them,
which is the test binary, and never into the library. cborinternal_p.h is present
only because
tst_tojson.cpp includes it for its own encode_half/decode_half reference, which
likewise runs in the test binary.
The fuzz oracle. Differential fuzzing needs upstream's implementation to compare
against. It is built as a standalone cbor-oracle binary and driven as a subprocess:
bytes in on stdin, pretty output and exit code out. An in-process oracle over FFI would have
been faster to write and would have made this whole page false.
tests/original/. Upstream's C++ test code, verbatim and hash-pinned. That is the point
of the exercise.
Checking it yourself
$ grep -rn "bindgen\|cc = \|build.rs" crates/ # nothing
$ find crates -name '*.c' # nothing
$ cargo tree # two crates, no dependencies
The one C file that ever existed in this project was the throwaway program that dumped struct offsets at kickoff. It was run outside the repository and only its output was committed.
The unsafe budget
unsafe is budgeted here, not sprinkled. The rule is structural rather than aspirational:
cbor-coreis#![forbid(unsafe_code)]. Notdeny, not a lint you can allow at a call site.forbidcannot be overridden further down the tree. The entire CBOR implementation lives under it.cbor-ffiis where everyunsafeblock lives, and each one carries a// SAFETY:line naming the invariant the caller must uphold. This is the crate that dereferences pointers handed to it by a C program, so the invariants are real and worth writing down.
Current count
$ grep -rn "unsafe {" crates/ --include='*.rs' | wc -l
80
Counting unsafe { counts blocks, which is the number that means something. A bare
grep -rn unsafe crates/ returns 101; the difference is 14 unsafe fn signatures and a
handful of lines of prose, including cbor-core's #![forbid(unsafe_code)] itself.
All 80 are in cbor-ffi:
| file | blocks |
|---|---|
parser.rs | 41 |
encoder.rs | 18 |
tojson.rs | 11 |
pretty.rs | 7 |
validation.rs | 3 |
cbor-core/ (any file) | 0 |
grep -rn SAFETY crates/ returns 92, so every block is accounted for with room to spare:
some invariants are stated once for a whole module and referred back to.
This started at zero when the shim was stubs and was published at each step rather than at the end. A count that only ever moves in the flattering direction is not worth much if nobody sees the intermediate values.
For scale: uv ships 73 unsafe blocks. Bun ships 13,044. Neither number is
damning on its own, since a runtime that talks to JavaScriptCore has different obligations
than a package resolver, but they bracket what "a lot" and "a little" look like in
shipped Rust.
Where the blocks are
The prediction made at kickoff was "pointer validation at every entry point, plus
_cbor_value_dup_string, and nothing else if the design holds". That is what happened.
Pointer validation at the boundary. Each of the 44 exported functions receives raw
pointers from C. The shim converts them to references once, on entry (as_ref and as_mut
in the parser, their equivalents elsewhere) and everything past that point is safe Rust
operating on &CborValue. This is the bulk of the 75, and it is why parser.rs has the
most: it has the most entry points.
Reading bytes out of the caller's buffer. be_load does a sized unaligned load at a
cursor the caller owns, and scan_subtree reads heads the same way while skipping a
subtree. The bounds check happens first, in safe code, every time; the // SAFETY: line
on each call site names which check established it. The subtree scan is where the count
last grew, from 75 to 80, and it is decision 16.
_cbor_value_dup_string. It allocates with libc malloc and hands ownership to the
caller, who frees it with free(). That is a genuine cross-language allocation contract and
there is no safe way to express it. It is the only place this library allocates on someone
else's behalf, and it is decision 12.
Not the unions. The layout parity page explains why the C unions are
modelled as a pointer-sized word rather than a Rust union: reading a union field is
unsafe regardless of whether it can misbehave, and spending blocks there would have
inflated this count without buying any safety.
Why publish it at all
Because the interesting failure mode of a C-to-Rust port is not "it does not compile". It is
a port that reaches memory safety by writing C in Rust syntax, with unsafe at every
awkward corner until the borrow checker stops objecting. The count, per crate, with a rule that the
core cannot contain any, is the cheapest available evidence that did not happen.
Verified 2026-08-02. make lint recomputes the count on every run.
Cookbook: porting a C library to Rust
This section is the part you can steal. It walks through how 0xicbor was actually built, in order, with the reasoning left in, including the bits that went wrong.
You do not need to know CBOR. You need to be able to read C well enough to follow a
switch, and to write Rust well enough to be annoyed by the borrow checker.
The one idea
Most "rewrite it in Rust" projects fail the same way: you rewrite the library, you rewrite the tests to match, and now you have no idea whether it behaves like the original. Every disagreement becomes a test edit, and a test suite you edited proves nothing.
The whole method here is one move that makes that impossible:
Keep the original test suite. Byte for byte. Make your Rust satisfy it.
Everything else (the ABI shim, the layout assertions, the symbol diff) exists to make that one move physically possible.
The order things happened
- Get to a red test loop: before writing a single line of real logic, make the original tests compile, link, run, and fail against an empty library. That failure count is your progress bar for the rest of the project.
- Match the ABI: the tests read struct fields directly, so your structs have to be laid out exactly like C's. This is measurable, not a matter of care.
- Write the first module: half-precision floats, by hand, in about sixty lines. Small enough to get right, and it has a test that proves it.
- Match printf exactly: where most of the remaining bugs live, and where I was wrong three times in a row.
- When the C is faster: green tests and 1.49x slower. Finding out which GCC optimisation pass you are competing with, and what the Rust equivalent of it is. Two wrong hypotheses before the right one.
What this costs you
Being honest about the trade: this method makes your Rust look less like Rust at the edges. You will write out-parameters and integer error codes because callers expect them. The trick is confining that to one thin layer and keeping the real implementation idiomatic underneath, which is what Why an ABI shim is about.
In exchange, every claim you make is a command someone else can run.
Step 1: get to a red test loop
The goal of the first few hours is not working code. It is a command that runs the original test suite against your empty library and prints a real number.
Once that exists, the rest of the project is turning red into green, and you always know exactly where you are. Without it, you write code for two days and then find out whether any of it works.
Pin the tests before you touch anything
Copy the upstream test directory in verbatim, then record what you copied:
$ cp -a ../tinycbor-upstream/tests/. tests/original/
$ cd tests/original
$ find . -type f -not -name hashes.txt | sort | xargs sha256sum > hashes.txt
Now sha256sum -c hashes.txt proves at any moment that you did not quietly "fix" a
test that was inconvenient. Run it at the end. Judges, reviewers, and future-you all
care about the difference between a suite you passed and a suite you edited.
Two details that matter more than they look:
- Generated files must land somewhere else. Qt's
mocwants to write next to the source; point it atbuild/instead, or your hash check fails on your own output. - Do this first. Pinning hashes after you have been working for a day proves nothing about the day you already had.
Find out what "all tests pass" even means
Build upstream once, unmodified, and run its own suite. That number is your ceiling:
$ tst_encoder 1596 passed
$ tst_parser 2506 passed
$ tst_tojson 827 passed
$ tst_cpp 2 passed
————
4931
Then read the tests and find the ones that cannot apply to a port. In this project
tst_cpp.cpp opens with:
#include "../../src/cborencoder.c"
#include "../../src/cborparser.c"
That is not a test of the library. It is a test that the C sources compile as C++. A
Rust port has no .c files for it to include, so the honest ceiling is 4,929, not
4,931, and the two missing rows get written down as a decision rather than silently
dropped.
Find these on day one. Discovering at hour 60 that your target was never reachable is a bad hour.
Link against nothing
Create the library crate and export every public symbol as a stub. Do not implement anything. The point is to make the linker happy:
$ nm -g --defined-only libtinycbor.a | wc -l
44
Then make the test binaries compile against your headers and link against your
archive. When make test runs and prints failures, you are done with step one.
One thing worth getting right: make the stubs return an error, not panic. A
staticlib built with panic = "abort" will kill the test process on the very first
call, and your baseline becomes "it crashed" instead of a number you can watch move.
#![allow(unused)] fn main() { // Not unimplemented!(): that ends the process and the baseline with it. const STUB: c_int = c_int::MAX; // CborErrorInternalError }
The number you are looking for
tst_encoder 2 passed 1594 failed
tst_parser 17 passed 2489 failed
tst_tojson 2 passed 825 failed
TOTAL 21 passed 4908 failed (of 4929)
Twenty-one passing with a library that does nothing at all. Those are tests that expect an error and coincidentally get one. That is the honest floor.
That is a good first day. You have written no logic, and you have a measurement.
Step 2: match the ABI
If the original tests call your library through a C header, then some of that header is not calls at all. It is code that reads your structs directly, compiled into the test binary, where your Rust has no say.
In tinycbor, 59 of 103 public functions are like this:
CBOR_INLINE_API CborType cbor_value_get_type(const CborValue *value)
{
return (CborType)value->type;
}
If CborValue::type sits at a different byte offset in your Rust struct than the
header believes, that returns a different byte. The failure surfaces as a wrong parse
result three layers away, and you will go looking for it in your parser.
So don't guess the layout. Measure it.
Get the numbers from C, once
Write a throwaway C program. Run it outside your repository so no C ends up in your tree, and commit only its output:
#define O(t, f) printf("offsetof %-12s . %-10s = %zu\n", #t, #f, offsetof(t, f))
printf("sizeof CborValue = %zu\n", sizeof(CborValue));
O(CborValue, parser);
O(CborValue, source);
O(CborValue, remaining);
sizeof CborEncoder = 32 sizeof CborParser = 16 sizeof CborValue = 24
offsetof CborValue . parser = 0
offsetof CborValue . source = 8
offsetof CborValue . remaining = 16
offsetof CborValue . extra = 20
offsetof CborValue . type = 22
offsetof CborValue . flags = 23
Then assert every one of those in a Rust test, so the tripwire trips at cargo test
instead of during a debugging session:
#![allow(unused)] fn main() { assert_eq!(size_of::<CborValue>(), 24); assert_eq!(offset_of!(CborValue, extra), 20); }
What to do about C unions
All three public structs contain something like this:
union { uint8_t *ptr; ptrdiff_t bytes_needed; CborEncoderWriteFunction writer; } data;
Rust has union, and using it is the literal translation. This port uses a single
pointer-sized field instead:
#![allow(unused)] fn main() { #[repr(transparent)] pub struct Word(pub *mut c_void); }
Every member of that C union is exactly pointer-sized and pointer-aligned, so the two layouts are identical, and the layout test proves that rather than assuming it.
The reason to prefer the plain word is the unsafe budget. Reading any field of a Rust
union requires unsafe, whether or not it can actually misbehave, because the
compiler cannot know which member is live. If you are publishing your unsafe count
as a quality signal, you do not want a dozen blocks that exist only to read a pointer
you already knew was there.
Which member is live is decided by the struct's own flags field, exactly as it is
in C. You have not lost any safety, because there was never any to lose.
Prove the symbols match
$ nm -g --defined-only target/release/libtinycbor.a \
| awk '$2 ~ /^[A-TV-Z]$/ {print $3}' | sort -u \
| diff reference-symbols.txt -
An empty diff means any program that linked against the original links against yours. That is the strongest statement a drop-in replacement can make, and it is one command.
Do this at the stub stage, before any real code exists. Symbol parity is a property of the interface, so it costs nothing to establish up front and then guards every commit afterwards. Finding a missing symbol on the last day is much worse than finding it on the first.
Step 3: write the first module
Pick the smallest self-contained thing in the library and do that first. Not because it unlocks the most tests (it usually unlocks none) but because it is where you find out whether your setup works before you have anything invested in it.
Here that was half-precision floats: IEEE 754 binary16, about sixty lines, no dependencies on anything else in the codebase.
Why not just add a crate
The half crate exists and is good. Using it would have been one line.
The rule for this project was no third-party CBOR or float crates, and the reason generalises past hackathon rules: wrapping a crate is not a port. If the interesting part of the module is delegated, you have not learned whether you can reproduce the original's behaviour. You have learned whether two libraries happen to agree.
It is also genuinely small. Sixty lines is not a heroic act.
The shape of it
A binary16 is 1 sign bit, 5 exponent bits, 10 mantissa bits. Going up to f32 is
always exact, because every half value fits. Three cases:
#![allow(unused)] fn main() { match exp { 0 if mant == 0 => f32::from_bits(sign), // zero 0 => { /* subnormal: renormalise */ } 0x1f => { /* infinity or NaN */ } _ => { /* normal: rebias by 127 - 15 = 112 */ } } }
The subnormal case is the only one with a loop: shift the mantissa left until the implicit bit appears, paying one exponent step per shift.
#![allow(unused)] fn main() { let mut m = mant; let mut shifts = 0; while m & 0x400 == 0 { m <<= 1; shifts += 1; } f32::from_bits(sign | ((113 - shifts) << 23) | ((m & 0x3ff) << 13)) }
One thing to know before you start: core has no powi. If you are writing
no_std, the arithmetic shortcuts (mant as f32 * 2f32.powi(-24)) are not available
and everything has to be shifts and masks. Better to know that at the top than to
write it twice.
Going down is where the bugs are
f32 to f16 is lossy, so it needs a rounding mode, and the right one is
round-to-nearest-even. Ties go to the even neighbour, not away from zero:
#![allow(unused)] fn main() { fn round_up(value: u32, shift: u32) -> u16 { let half = 1u32 << (shift - 1); let rest = value & ((1u32 << shift) - 1); let odd = (value >> shift) & 1; u16::from(rest > half || (rest == half && odd == 1)) } }
If you get this wrong, almost everything still works. Only exact ties differ, and exact ties are rare in casual testing and guaranteed in a fixture suite.
The test that makes it free
Half-floats have 65,536 possible values. That is small enough to check all of them:
#![allow(unused)] fn main() { #[test] fn round_trips_exactly() { for h in 0u16..=0xffff { if is_nan(h) { continue; } // NaN payloads are not preserved bit-for-bit assert_eq!(encode(decode(h)), h, "half {h:#06x} did not round-trip"); } } }
This runs in under a millisecond and it is a complete proof for one direction. When your input domain is small, enumerate it. Property tests and fuzzing are for when you cannot.
Then pin the boundaries separately, because a round-trip test will not catch a consistently wrong rounding mode:
#![allow(unused)] fn main() { assert_eq!(decode(0x7bff), 65504.0); // largest finite half assert_eq!(decode(0x0001), 5.9604645e-8); // smallest subnormal assert_eq!(encode(2049.0), encode(2048.0)); // tie rounds to even assert_eq!(encode(70000.0), 0x7c00); // overflow saturates to infinity }
Sixty lines of implementation, four tests, and you never think about it again. That is what you want from the first module.
Step 4: match printf exactly
Once the parser worked, 2,196 tests still failed. Almost none of them were parser bugs. The tests stringify what they parsed and compare it to a fixture, and the stringifier was still a stub, so every one of them reported:
FAIL! : tst_Parser::arrays(0) '!err' returned FALSE. Got error ""; decoded stream:
An empty error message and an empty stream. Two stubs, thousands of failures. Worth remembering when a failure count looks catastrophic: cluster the failures before you start fixing them. One missing function can account for half your red.
The part Rust cannot do for you
C prints doubles with %.17g. Rust has no equivalent, and the difference is not
cosmetic. It is compared against fixtures character by character.
The rule, from C99 §7.19.6.1, is short. Let X be the base-10 exponent:
- If
-4 <= X < precision, print%f-style withprecision - 1 - Xdigits after the point. - Otherwise print
%e-style withprecision - 1digits. - Either way, strip trailing zeros, and the point if nothing survives.
The trick that makes it easy in Rust is that {:e} hands you X for free:
#![allow(unused)] fn main() { let sci = format!("{:.*e}", precision - 1, v); let (mantissa, exponent) = sci.split_once('e').unwrap(); let x: i32 = exponent.parse().unwrap(); }
Then follow the rule literally. Don't be clever; the rule is the specification.
Do not write the expected values from memory
This is the actual lesson of this chapter. My first test looked like this:
#![allow(unused)] fn main() { assert_eq!(format_g(1e-4, 17), "0.00010000000000000000"); }
It failed. I assumed my implementation was wrong. It was not. My expectation was.
%.17g of 1e-4 is 0.0001, because trailing zeros come off after the %f
formatting, not before.
I got three of these wrong in a row (1e-4, 3.14159, 9.8765) before doing the
obvious thing:
$ python3 -c "print('%.17g' % 1e-4)"
0.0001
Python's % formatting delegates to the same C rules. Thirty seconds of generating
ground truth would have saved three rounds of second-guessing correct code.
Generate your fixtures from the thing you are trying to match. Not from your memory of what it does. If you are porting from C, you have a C compiler and a shell right there.
The switch points are where the interesting cases live, so pin those specifically:
#![allow(unused)] fn main() { assert_eq!(format_g(1e16, 17), "10000000000000000"); // %f side assert_eq!(format_g(1e17, 17), "1e+17"); // %e side assert_eq!(format_g(1e-4, 17), "0.0001"); // %f side assert_eq!(format_g(1e-5, 17), "1.0000000000000001e-05"); }
The other classic: UTF-8 escaping
The second half of this module escapes strings, and it has one detail worth stealing. Upstream's condition for "printable, emit as-is" is:
if (uc < 0x7f && uc >= 0x20 && uc != '\\' && uc != '"')
That is < 0x7f, not <= 0x7f. So DEL gets escaped to . It is the kind of
boundary you will reproduce wrongly if you translate the intent ("printable ASCII")
instead of the code.
Translate the code. Write the comment about intent next to it.
#![allow(unused)] fn main() { // Note 0x7f is escaped too: the condition upstream is `< 0x7f`, not `<= 0x7f`. _ if (0x20..0x7f).contains(&uc) => out.push(ch), }
Astral characters are the other one: upstream emits a surrogate pair, so 😀 becomes
😀 rather than one ὠ0. Rust's char is a full code point, so you
have to split it back apart yourself:
#![allow(unused)] fn main() { let hi = (uc >> 10) + 0xd7c0; let lo = (uc % 0x400) + 0xdc00; }
Neither of these is hard. Both are invisible until a fixture disagrees with you.
Step 5: when the C is faster than your Rust
The suite went green at 4,929 of 4,929 and the port was 1.49x slower than the library it replaced. Not on one pathological input, but on all eight benchmark files, from 1.23x to 1.83x, with p99 tracking p50 so it was systematic rather than tail noise.
It ended up 2.8x faster on all eight. This chapter is how, and it is mostly about the hypotheses that were wrong on the way, because there were more of those than there were fixes.
First: is it one thing or everything?
A 1.5x spread across eight structurally different documents usually means one shared cost, not eight separate ones. The cheapest way to test that is to delete the suspect and see what happens, even if the deletion is not something you could ship.
The suspect was the source dispatch. Upstream lets a caller supply their own reader instead of a flat buffer, and every read checks which it is:
#![allow(unused)] fn main() { fn can_read(it: &CborValue, len: usize) -> bool { if external(it) { return (ops(it).can_read_bytes)(it.source.0, len); } (end(it) as usize).saturating_sub(ptr(it) as usize) >= len } }
external is (*it.parser).flags & EXTERNAL_SOURCE != 0. So: hardcode it to false,
rebuild, re-measure, throw the build away.
| corpus | with the branch | branch compiled out |
|---|---|---|
map_heavy | 1.83x | 1.10x |
tagged | 1.58x | 1.07x |
text_utf8 | 1.70x | 1.11x |
flat_array | 1.18x | 0.99x |
One branch, essentially the whole regression. That is the answer to where. It is not the answer to why, and the difference matters, because upstream has the identical branch in the identical place and is not slow.
Wrong hypothesis one: strict aliasing
The obvious story: C has type-based alias analysis. it->source.ptr is a uint8_t *
and it->parser->flags is a uint32_t, different types, so a C compiler may assume a
write to one cannot touch the other and hoist the flag load out of the loop. Rust
emits no TBAA metadata at all, so LLVM must reload after every cursor write.
It is a tidy story. It is also testable in about three minutes: rebuild upstream with
-fno-strict-aliasing and re-run the same benchmark.
mean ratio: 0.994
Nothing. Not a small effect. No effect at all. If the story had been right, taking TBAA away should have cost C most of its advantage.
Test the explanation, not just the fix. The fix would have been the same either way, and I would have shipped a confident, plausible, wrong paragraph about it.
Wrong hypothesis two: it is just inlining
Next observation, found by accident: a plain -O2 build of upstream is 1.66x slower
than the -O3 build the benchmark compares against. Whatever C is getting, it is
getting from -O3. -O3 mostly means more aggressive inlining, so: inlining, then.
Two more builds:
| upstream build | parse p50 vs stock -O3 |
|---|---|
-O2 -finline-functions | 1.34x slower |
-O3 -fno-unroll-loops -fno-tree-loop-vectorize | 1.03x |
-O3 -fno-ipa-cp-clone | 1.17x slower |
Inlining alone does not recover it. Unrolling and vectorisation are worth almost
nothing here. But -fipa-cp-clone, interprocedural constant propagation with
cloning, is worth 17% on its own.
That pass makes a specialised copy of a function for a call site where an argument is constant, then folds the constant through the copy. Applied to a parser whose reads all branch on one flag, it produces a version of the read path with the branch gone.
Which is monomorphisation. GCC was doing it in the optimiser.
The Rust version of that pass is the type system
rustc will not speculate on a runtime value, and it should not, because that is a heuristic, and heuristics are exactly what a language with generics does not need. The transformation is available; it just has to be asked for.
So the four source operations became a trait, with the two sources as types:
#![allow(unused)] fn main() { trait Source { fn can_read(it: &CborValue, len: usize) -> bool; unsafe fn read(it: &CborValue, offset: usize, len: usize) -> u64; fn advance(it: &mut CborValue, n: usize); fn transfer_string(...) -> c_int; } struct Buffer; // CborValue::source is the cursor, CborParser::source is the end struct Reader; // CborValue::source is a token, CborParser::source is a vtable }
Every internal function took S: Source, and each exported entry point picked an
instantiation exactly once:
#![allow(unused)] fn main() { macro_rules! dispatch { ($it:expr, $f:ident($($arg:expr),* $(,)?)) => { if external($it) { $f::<Reader>($($arg),*) } else { $f::<Buffer>($($arg),*) } }; } }
The Buffer instantiation now contains no branch and no indirect call anywhere in it.
Mean ratio 1.492 to 1.038, three of eight files faster than C, for 4,128 bytes of
extra .text. Three more rounds follow.
Note what did not change: the flag still lives in CborParser::flags, at the offset
the C header specifies, set by the same function that always set it. The ABI cannot
tell the difference. Specialising on a value that has to stay where C put it is fine
as long as you only read it where C can see you reading it.
And one more wrong idea, measured before believed
advance_recursive recurses once per nesting level, and at each level it builds a
CborValue that enter_container immediately overwrites in full. Obvious waste, and
the fix is the more idiomatic Rust: return the value instead of filling an
out-parameter. It is what the rest of this port does, and it is in the decision log as
a principle.
On the nesting-heavy corpus file it went from 1.25x to 1.68x.
Three words returned by value go through the return slot on every one of 160,000 calls; filled in place, they were already where the next call wanted them. The out-parameter stayed, and the reasoning is decision 14 so that nobody, including me in a week, tidies it back.
Round two: read the generated code, then the profile
Mean 1.038 was better than 1.492 and still not a win. Two more rounds got it to 0.984, and neither came from having another idea. Both came from looking.
The disassembly. advance_recursive opened with the obvious spelling:
#![allow(unused)] fn main() { if is_fixed_type(it.type_) { return advance_internal::<S>(it); } if !is_container(it.type_) { /* string */ } }
LLVM compiled is_fixed_type into a rotate and a bit-table lookup against a
constant. Thirteen instructions and three branches, before any work, in a function
that runs 504,000 times on the nesting corpus. GCC compiled the same logic into
three instructions and one branch, because it noticed what I had not: the classes
being tested are each a pair of major types differing only in bit 5. Byte string
is 0x40 and text is 0x60; array is 0x80 and map is 0xa0. Mask that bit off and each
pair collapses to one comparison.
#![allow(unused)] fn main() { match it.type_ & !MAJOR_PAIR_BIT { TYPE_BYTE_STRING => …, // both string types TYPE_ARRAY => …, // both container types _ => return advance_internal::<S>(it), } }
Five instructions, two branches. The clearer source was the slower one, and the faster one is arguably clearer once the comment explains the bit.
While in there: the string branch sets up a six-argument call, and every register
that call needs is one advance_recursive must push on entry and pop on exit. On a
function that recurses once per nesting level, a branch taken only at the leaves was
charging its register pressure to the whole descent. Behind #[inline(never)] the
frame went from 48 bytes to 32.
The profile. callgrind on the map-heavy corpus put 44% of parse time in
iterate_string_chunks, where we ran 16.4 M instructions to the C's 13.6 M. The C
symbol was named iterate_string_chunks.constprop.0.
That suffix again. The walker takes what-to-do-with-each-chunk as an argument, so
the branch lives inside the chunk loop, and GCC had cloned it per call site. Exactly
the same mistake as the byte source, one level down, and it had been sitting there
the whole time. Three types implementing one Op trait, and the choice moves to the
call site.
That last one took the mean under 1.0.
| mean | files faster than C | |
|---|---|---|
| before any of this | 1.492 | 0 of 8 |
| monomorphise the source | 1.038 | 3 of 8 |
| fix the dispatch, outline the string branch | 1.014 | 4 of 8 |
| monomorphise the chunk operation | 0.984 | 5 of 8 |
Round three: stop competing on their terms
0.984 is parity, and parity was where three rounds of careful tuning had left it. Each round had made the same move: find where the C's compiler was doing something rustc was not, and do it by hand. That works right up until you have caught up, and then it stops, because you are running the same algorithm.
So the last round asked a different question. Not "why is their code faster here" but "what is this code actually for".
cbor_value_advance skips the item under the cursor and everything nested inside it.
Upstream walks that subtree recursively, decoding each item it passes into a
CborValue. Then it discards all of it. Nothing survives the walk except where the
cursor ended up and whether anything was malformed.
Which means the decoding is work nobody reads. And it is not a rounding error: on a flat array of integers it was 106 instructions an item in this port and 117 in the C, where the walk itself needs about ten.
The replacement is one flat loop that reads heads, adds lengths, and keeps a small stack of how many items each open container still owes. Nesting turns out not to matter when you are only skipping: a container of N items is just N more items to get through.
| mean | files faster than C | |
|---|---|---|
| before any of this | 1.492 | 0 of 8 |
| monomorphise the source | 1.038 | 3 of 8 |
| fix the dispatch, outline the string branch | 1.014 | 4 of 8 |
| monomorphise the chunk operation | 0.984 | 5 of 8 |
| scan instead of descend | 0.362 | 8 of 8 |
The three tuning rounds together were worth 1.49x to 0.98x. Asking what the function was for was worth 0.98x to 0.36x.
The part that makes it safe to do at all
Rewriting a traversal in a port is exactly the kind of change that quietly breaks behaviour, because the errors are the hard part. CBOR has a specific taxonomy of them and the tests check which one comes back, not just that something did.
So the scan never reports an error. It hands back to the recursive code the moment it meets anything it does not want to reason about: a malformed head, a length the original would reject, nesting past 64 levels, a break where one may not go. Every error still comes from the original path, in the original order. The scan can be right or it can be absent; it cannot be subtly wrong about what went wrong.
That property is what let this ship. It also means the check is easy to state: replay
inputs through cbor_value_advance against both libraries, compare the error code, the
final cursor offset and the resulting type. Across the fuzz corpus, the regression
corpus and the benchmark corpus, 3,980 inputs, zero differences.
Building that check found a bug that had nothing to do with the scan, and predated it
by weeks: upstream copies the cursor into the caller's next before walking a string
and works on that, so a failed walk still says how far it got, while this port walked a
local copy and wrote it back only on success. Thirty-seven inputs disagreed. Same shape
as the pretty printer's missing copy_current_position from an earlier chapter: right
error, wrong state left behind, invisible to anything that only checks return values.
Two rounds of that now. When a port is wrong in a way tests do not catch, it is usually about what it leaves behind rather than what it returns.
What to take from this
- Delete the suspect before optimising it. A build you throw away answers where in minutes.
- A hypothesis that explains the numbers is not the same as the one that is true. Both of mine did, and both were wrong. Compiler flags are a cheap way to interrogate the other language's implementation directly.
- When C beats you, ask which pass is doing it.
-fno-your way through the ones that fit the shape of the code. The answer is often a specific, nameable transformation rather than "C is closer to the metal". - Idiomatic is a prior, not a proof. Measure the idiomatic version too.
- Once you have found one instance of a pattern, grep for the rest of it. The chunk walker had the identical defect as the byte source and survived two more rounds of tuning because I was looking at the thing I had just changed.
- Read the disassembly of the function you think is hot. Not to hand-optimise it, but because the compiler will show you which of your source constructs it did not like, and that is a much shorter list than the things you could try.
- Matching the original's algorithm gets you to parity, and then it stops. Parity is the ceiling of transliteration. Going past it means asking what the function is for, which is a question about the API and not about the code.
- When you do depart from the original, build the departure so it can only be right or absent. A fast path that never reports an error cannot get the error taxonomy wrong, and that is most of what there is to get wrong in a port.
And publish the regression while it is still a regression. This port shipped a 1.49x number in its README for a day. That is what made it worth fixing.
Building
One command. It builds the static library and the original test binaries.
$ make
Requirements
- A Rust toolchain (2021 edition,
offset_of!needs 1.77 or newer) - A C++ compiler with C++20, for upstream's Qt tests
- Qt6 Test, discovered through
pkg-config moc, from the same Qt installation
Nothing else. The Rust side has no dependencies at all, which you can check the same way anyone else would:
$ cargo tree
Qt is only needed to run upstream's tests. If all you want is the library,
cargo build --release produces libtinycbor.a on a machine with no Qt at all.
Targets
make | library plus the original test binaries |
make test | the original suite plus this port's own differential tests |
make test-tools | cbordump and json2cbor against upstream's binaries |
make symbols | diff exported symbols against upstream's library |
make lint | clippy with warnings denied, plus the unsafe budget check |
make fmt | rustfmt --check |
make fuzz | differential fuzz; DURATION=900 TARGET=encode_diff to pick |
make bench | rebuild the drivers and rewrite bench/results.json |
make clean | remove build/ and target/ |
make test needs Qt6. Everything else runs from a fresh clone with nothing beside it:
make bench and make fuzz both fall back to bench/reference/libtinycbor-upstream.a,
which is committed, so the C side of every comparison is already here.
make test-tools is the exception. It compares against upstream's cbordump and
json2cbor binaries, and binaries are not vendored, so without a checkout it says so and
skips rather than fails. Troubleshooting has the exact commands if you
want that comparison too.
Overriding tools
$ make CXX=clang++ MOC=/usr/lib/qt6/libexec/moc
Verifying the suite is untouched
$ cd tests/original && sha256sum -c hashes.txt
This should stay clean after a full build. Generated moc output goes into build/,
never next to the test sources.
Fresh clone
The repository builds without tinycbor checked out beside it. The C headers the tests
compile against are vendored, and upstream's reference library in bench/reference/ is only
a comparison target.
$ git clone https://github.com/keirsalterego/0xicbor && cd 0xicbor && make test
Next steps
- Using the library once it has built.
- Troubleshooting if it has not.
- Contributing before you change anything.
Verified 2026-08-02 on Debian-based Linux x86-64, Qt 6, rustc 1.97.
Troubleshooting
Things that go wrong, in roughly the order people hit them.
make says it cannot find Qt6Test
Package Qt6Test was not found in the pkg-config search path
The original test suite is Qt/C++, so building it needs Qt6 Test and moc. On Debian and
Ubuntu:
$ sudo apt install qt6-base-dev qt6-base-dev-tools pkg-config
moc moves around between distributions. If the makefile cannot find it, point at it:
$ make MOC=/usr/lib/qt6/libexec/moc
The library itself does not need Qt for anything. cargo build --release gives you
libtinycbor.a with no Qt installed at all. Qt is only there so upstream's tests can run
unmodified, which is the entire premise of the project.
make test-tools says it skipped
missing /home/you/tinycbor-upstream/build-test/tools/cbordump
(skipped: no upstream tools to compare against)
That check compares this port's cbordump and json2cbor against upstream's binaries.
Binaries are not vendored the way bench/reference/libtinycbor-upstream.a is, so you need
upstream checked out and built:
$ git clone https://github.com/intel/tinycbor ~/tinycbor-upstream
$ cd ~/tinycbor-upstream && git checkout 9441b2ca
$ cmake -B build-test && cmake --build build-test
Point somewhere else with UPSTREAM=/path/to/build/tools.
A skip is not a failure. The rest of make test is unaffected, and a real divergence exits
non-zero and stops the build.
The fuzzer will not start
no oracle at .../fuzz/oracle/cbor-oracle -- run fuzz/oracle/build.sh
The differential oracle is upstream's C compiled into a standalone binary, and fuzz/run.sh
builds it for you. It does not need the checkout above: without one it compiles
fuzz/oracle/cbor-oracle.c against the vendored headers and the committed
bench/reference/libtinycbor-upstream.a, and says which of the two it used. A checkout
still wins when you have one, since pointing TINYCBOR= at a different upstream commit is
the only way to fuzz against one.
You also need nightly Rust and cargo-fuzz:
$ rustup toolchain install nightly
$ cargo install cargo-fuzz
The oracle runs as a subprocess, never linked in. That is not an implementation detail you can optimise away, it is the thing that keeps "this library contains no C" checkable. See Where the C ends.
make bench refuses, or warns about a changed archive
!! rust archive changed since link; rerun bench/build.sh to measure the current tree
The benchmark records the SHA-256 of both archives when it links its drivers. If you rebuild the library afterwards, the recorded numbers no longer describe the code you have, and the harness says so instead of quietly publishing stale figures.
$ ./bench/build.sh && python3 bench/run.py
Also: run it on an idle machine. The numbers in the repository were re-taken because a
run went out at load 6.4 on 8 cores, and it was visibly wrong. The recorded load average is
in results.json so you can tell whether to believe a given run.
Linker errors about pthread_* or dlsym
undefined reference to `pthread_mutex_lock'
Add -lpthread -ldl. Linking a Rust staticlib pulls in the Rust standard library, which
needs both. This is the one thing that is genuinely different from linking the C, and it is
not optional.
The suite passes but sha256sum -c fails
Something wrote into tests/original/. Nothing in this repository is supposed to: moc
output goes into build/, and the makefile is set up that way deliberately.
$ git status tests/original
$ git checkout tests/original
If it keeps happening, you are probably running upstream's CMake inside the checkout rather than this repository's makefile.
cbor_value_* returns garbage, or crashes
Check the lifetimes first. CborValue holds a pointer back to the CborParser it came
from, and nothing enforces that the parser outlives it. A parser declared in an inner scope
and a value used after that scope closes is the classic version, and it will usually appear
to work for a while.
This is inherited from the C API and cannot be fixed without changing the ABI, which is the one thing that cannot change. Decision 21 says so at more length.
Everything builds but the numbers do not match the docs
Fair. Every figure in these docs comes from a run on one laptop, and the pages that carry numbers say when they were verified. Re-run and trust your own output:
$ make test # 4,929 / 4,929
$ make symbols # 44/44, zero diff
$ make lint # includes the unsafe budget count
$ python3 bench/run.py
If your numbers differ and you think the docs are wrong rather than your machine, that is worth an issue.
Next steps
- Building for the full target list.
- Contributing if you are about to change something.
Contributing
The rules here are unusual, because the project is unusual. It is a port judged against a test suite it is not allowed to touch, and most of what follows exists to keep that claim honest.
The four things that are not negotiable
tests/original/ is never edited. Not to fix a test, not to skip one, not to adjust
whitespace. The hashes are pinned in tests/original/hashes.txt and verified at ship. If a
test genuinely cannot pass, it stays failing and gets an entry in
the decision log. A 94% pass rate you can reproduce beats a 100% claim you
cannot.
The shipped library contains no C. No cc crate, no bindgen, no build.rs compiling
anything, nothing under crates/ linking against upstream. The fuzz oracle is upstream's C
in a separate executable spoken to over a pipe. If you find yourself wanting to link it in
for speed, the answer is no, and Where the C ends
explains why.
No third-party CBOR crate. Not ciborium, not serde_cbor, not minicbor, not half.
Wrapping an existing crate is not a port. cbor-core uses core and alloc and nothing
else.
cbor-core stays #![forbid(unsafe_code)]. Every unsafe block lives in cbor-ffi
and carries a // SAFETY: line naming the invariant the C caller must uphold. make lint
recomputes the count and fails if a block loses its line, so this is enforced rather than
requested.
Before you open a PR
$ make test # 4,929 / 4,929, and the port's own differential tests
$ make lint # clippy, the unsafe budget check, and relative links
$ make fmt # rustfmt --check
$ make symbols # 44/44 exported, zero nm diff against upstream
All four, green. If you touched the parser or the encoder, also run the fuzzer for longer than the default minute:
$ TARGET=encode_diff DURATION=900 make fuzz
Sixty seconds is enough to say you fuzzed. It is not enough to find anything. Both real divergences found so far needed either fifteen minutes or a target that had never run before.
run.sh appends a row to fuzz/history.tsv when it finishes, so a long run you did is
recorded even though the log it wrote gets overwritten by the next one. Leave that row in
your PR; it is the only evidence the run happened.
make lint also resolves every relative link in the book, readme.md and decisions.md,
and fails on the first one pointing at a file that is not there. mdBook turns a broken link
into a 404 without complaining, so a rotted page looks exactly like a healthy one until
somebody clicks it.
Changing behaviour
The equivalence claim is against upstream at commit 9441b2ca. That has a consequence
people find surprising: if upstream has a bug, this port keeps it.
Two are documented. cbor_value_to_json_advance does not validate UTF-8 although its own
header says it does, which is
intel/tinycbor#331 and
decision 17. cbor_encode_simple_value accepts simple value 24 and emits
f8 18, which upstream's own parser then rejects, which is
decision 22.
Fixing either would turn a passing differential fuzz into a failing one and make the central claim of the project false. If upstream takes a fix, this follows. Until then, a divergence would be ours, not theirs.
So: a PR that makes this port better than upstream is a PR that has to argue its case in the decision log first.
Writing style
The code is read by people, so it is written for them.
- Comments explain why, and name the spec section when there is one (
RFC 8949 §3.2.1). - A comment that restates the line below it gets deleted.
- Plain engineering voice. No "this function is responsible for", no "robust", no "leverages".
Result<T, CborError>and?, never a translated errno. Where a C function returns an out-parameter plus a status, the Rust one returns the value.
The decision log is the other half of this. Every non-obvious divergence gets an entry at the moment you make it. Reconstructing them later is how you end up with ten empty bullets, and it shows.
Commits
Conventional Commits, scoped to the module:
fix(parser): leave the cursor where a failed string walk stopped
test(port): differential test for the caller-supplied reader source
docs(decisions): the upstream UTF-8 bug, and why this port keeps it
Scopes in use: parser, encoder, ffi, pretty, tojson, cbordump, json2cbor,
port, original, fuzz, bench, docs.
Write the body as though the reader has the diff but not your afternoon. What was wrong, what it did to a caller, how it was found. The commit log is the best record of the work this project has.
Adding a test
Tests for what upstream's suite does not reach go in tests/port/, and they follow one
pattern: the same source is built twice, once against each archive, and the two
transcripts are diffed.
No expected-output file. An expected-output file is a snapshot of what the code did on the day you wrote it, and it drifts into being a snapshot of a bug. Diffing against upstream cannot drift, because upstream is the specification here.
That pattern has found five bugs so far, none of which any fuzzer would have reached, and decision 18 is the write-up of the worst of them.
Next steps
- Building for the target list.
- Troubleshooting when a gate fails for environmental reasons.
- The decision log before you argue with any of the above.
Decisions
Every place this port diverges from a literal transliteration of upstream, and why. Written as the decisions were made, not reconstructed afterwards.
1. The port is a static library with a byte-identical C ABI
Upstream's test suite is Qt/C++ calling the C API. There is no honest way to run it
unmodified except to hand it a libtinycbor.a it cannot distinguish from the real one.
So cbor-ffi is a crate-type = ["staticlib"] that exports the same 44 symbols with the
same signatures, and the tests link against it with no edits and no shim layer of our own.
The alternative, rewriting the tests to call a Rust API, would have made the pass rate meaningless. A test suite you rewrote is a test suite you can make pass.
Cost: the port carries the shape of a C API into Rust. cbor_value_get_int_checked returns
its result through an out-parameter because callers expect that. Inside cbor-core the
same operation returns Result<i64, CborError>; the out-parameter convention exists only
in the shim, in one place.
2. tests/original/hashes.txt is pinned, and tst_cpp is left failing
The 14 files under tests/original/ are upstream's at commit 9441b2ca, byte for byte,
with SHA-256 recorded at kickoff. sha256sum -c hashes.txt is re-run at ship time.
tst_cpp.cpp cannot pass and will not be made to. It opens with:
#include "../../src/cborencoder.c"
#include "../../src/cborparser.c"
...
It is not a test of the library. It is a test that upstream's C sources compile cleanly as
C++. A Rust port has no .c files for it to include, so the test is inapplicable by
construction rather than failing on behaviour. It contributes 2 rows, which is why the
reachable total is 4,929 of upstream's 4,931 and not 4,931.
Excluding it from the makefile rather than letting it fail to compile is a presentation choice; the reason is recorded here so the missing 2 rows are not mistaken for a gap.
3. tst_c90 is kept, and it constrains the headers
tst_c90.c includes only cbor.h and compiles under -std=c90 -pedantic. It survives the
port because it tests the header, which we still ship. It is a genuine constraint: the
vendored headers must stay C90-clean, which rules out tidying them up with newer C.
4. The C headers are vendored; the library contains no C
crates/cbor-ffi/include/ holds upstream's cbor.h, cborjson.h, cborinternal_p.h,
compilersupport_p.h and the two generated tinycbor-*.h. They are vendored so a fresh
clone builds without upstream checked out beside it.
This is worth being precise about, because "no C" is a claim judges should be able to check.
libtinycbor.a is compiled entirely from Rust: no cc crate, no bindgen, no linking
against upstream. The headers are the ABI contract, and the 59 static inline accessors in
cbor.h compile into whatever program includes them, which is the test binary, never into the
library. cborinternal_p.h is vendored only because tst_tojson.cpp includes it for its
own encode_half/decode_half reference, which likewise runs in the test binary.
5. The fuzz oracle is a separate process, not FFI
Differential fuzzing needs upstream's C implementation to compare against. It is built as a
standalone cbor-oracle binary and driven as a subprocess: bytes in on stdin, pretty
output and exit code out. Nothing about it is linked into the Rust artifact.
An in-process oracle via FFI would have been faster and simpler, and would have quietly made rule "no C in the shipped artifact" false. Naming it here so nobody has to guess.
6. Unions are modelled as a pointer-sized word, not a Rust union
All three public structs contain a C union:
union { uint8_t *ptr; ptrdiff_t bytes_needed; CborEncoderWriteFunction writer; } data;
Every member of every one of these unions is exactly pointer-sized and pointer-aligned, so a single opaque word is layout-identical, and the layout test proves it rather than assuming it.
A real union would have been the more literal translation, but reading any field of a
Rust union is unsafe regardless of whether it can actually misbehave. Since unsafe in
this port is a published number, spending blocks on field reads that carry no risk would
inflate the count without buying safety. Which member is live is decided by the owning
struct's flags, exactly as in C.
7. Stubs return CborErrorInternalError rather than unimplemented!()
During scaffolding every entry point is a stub. Panicking would be the idiomatic Rust
placeholder, but the library is built with panic = "abort", so the first call kills the
Qt test process and the baseline becomes "it crashed" instead of a number.
Returning CborErrorInternalError (INT_MAX) lets the whole suite run and report per-row
results, which turns the failure count into a progress bar. Baseline was 21 passed, 4,908
failed, the 21 being rows that expect an error and coincidentally get one.
8. cbor-core is no_std + alloc; cbor-ffi is not
Upstream targets microcontrollers. Dropping that would be a real reduction in what the
library can do, so cbor-core is #![no_std] and #![forbid(unsafe_code)], with alloc
pulled in only for the dup_string family and indefinite-length string reassembly,
both of which are optional upstream too.
cbor-ffi deliberately uses std. It only ever links into a hosted C program, and std
supplies the allocator and panic handler that a no_std staticlib would otherwise need
hand-rolled. Hand-rolling both would have meant a #[global_allocator] forwarding to libc
malloc, which is more unsafe and more moving parts for no benefit at the only place
this crate is ever used. The portability claim belongs to cbor-core, where it pays.
9. CborNoError has no Rust variant
Upstream's CborError reserves 0 for success and every function returns it. In
cbor-core success is Ok(()), so the enum carries only failures and starts at 1. The
discriminants of the failure variants are still fixed by the C ABI, gaps and all. Upstream
groups them in blocks of 256 by category so callers can range-check a class of failure, and
that numbering is reproduced exactly.
The Result boundary is the shim. cbor-core never sees an integer error code, and the
FFI layer never sees a Result.
10. The suite is compiled directly, not through upstream's CMake
Upstream drives its tests from CMake through a tinycbor_add_qtest() helper in
cmake/TinyCBORHelpers.cmake. Reproducing that would mean carrying a slice of upstream's
build system into this repo for no gain: all the helper does that matters here is run moc
and link Qt Test, which is four lines of makefile.
The test sources themselves are still untouched. moc output is written into build/ so
that tests/original/ stays clean enough to hash-verify.
11. Upstream has moved from qmake to CMake
Worth noting because it dates any guide written against tinycbor. There are no .pro files
at 9441b2ca; the suite is CMakeLists.txt throughout. The port targets the current tree,
not the qmake-era one.
12. The only C symbols we call are libc's fwrite, malloc and free
The rule is no source-language runtime: a C-to-Rust port must not FFI back into the
library it replaces. This port does not. cargo tree lists four crates and they are all
in this repository (cbor-core, cbor-ffi and the two tools) with no third-party
dependency at any depth, no build.rs, no cc and no bindgen. libtinycbor.a is
compiled entirely from Rust. The differential fuzz oracle is upstream's C built as a
standalone binary and driven as a subprocess over a pipe.
There are exactly two extern "C" blocks in the tree, both libc, both forced by the ABI
rather than chosen.
#![allow(unused)] fn main() { extern "C" { fn fwrite(ptr: *const c_void, size: usize, nmemb: usize, stream: *mut c_void) -> usize; } }
The signature we have to implement is:
CborError cbor_value_to_pretty_advance(FILE *out, CborValue *value);
FILE is an opaque libc type owned by libc. A caller hands us one it opened, and the only
way to write to it is to call libc. Going through fileno() and write() would still be
libc, just less direct.
#![allow(unused)] fn main() { extern "C" { fn malloc(size: usize) -> *mut c_void; fn free(ptr: *mut c_void); } }
_cbor_value_dup_string hands the caller a block and the documentation says to release it
with free(). That names the allocator. Returning a pointer from Rust's global allocator
would be a heap mismatch the moment anyone honoured the contract. It happens to be malloc
on this target today, and Rust promises nothing about that continuing to be true.
This is not a loophole, because it does not get us anything. Rust's std links libc on
Linux in every program ever compiled; if the rule barred that, no Rust port could exist on
this platform. What the rule is actually about, reusing the original implementation instead
of rewriting it, is not happening anywhere here, and the empty nm diff against a library
built from entirely different source is the evidence.
cbor-core, which is the whole CBOR implementation, has no extern blocks at all and is
#![no_std].
13. The byte source is a type parameter, not a flag test
Upstream reads parser->flags & CborParserFlag_ExternalSource inside each of the four
source operations (can_read_bytes, read_bytes, advance_bytes, transfer_string)
so the test runs on the head of every item. Transliterating that cost 1.49x against the C
on an eight-file corpus, and it was almost the whole gap: hardcoding the branch to the
buffer case took map_heavy from 1.83x to 1.10x.
The interesting part is why the same code is not slow in C. GCC at -O3 runs
-fipa-cp-clone, which clones a function specialised on a constant argument and folds the
branch out of the clone. Building upstream with -fno-ipa-cp-clone and changing nothing
else costs it 17%. -fno-strict-aliasing, which was my first guess, costs it nothing
measurable (0.994x), so this is not a TBAA story, it is a specialisation story.
rustc has no equivalent and will not grow one: it does not speculate on runtime values.
What it does have is monomorphisation, which is the same transformation with the decision
moved from the optimiser to the type system. So the four operations became a Source
trait with a Buffer and a Reader impl, the parser internals take S: Source, and each
#[no_mangle] entry point picks an instantiation once. The buffer instantiation contains
no branch and no indirect call anywhere in it.
The ABI is untouched: the flag still lives in CborParser::flags exactly where the header
says, and cbor_parser_init_reader still sets it. It is read once per API call now instead
of once per byte read.
Cost: two copies of the parser, which measured 4,128 bytes of .text on the benchmark
driver, or 0.6%. Mean ratio went 1.492 to 1.033, and three of the eight corpus files
are now faster than the C.
The alternative was threading a bool down by hand, which is the same specialisation
written out longhand and would have doubled the argument list of every internal function
without the compiler checking that no call site got it wrong.
14. enter_container keeps C's out-parameter, against the grain of the rest
Everywhere else in this port, a C function that returns a value through an out-parameter
plus a status code becomes a Rust function that returns the value. enter_container is the
exception: internally it still takes &mut CborValue and fills it.
The idiomatic version was written and measured. advance_recursive recurses once per
nesting level, so on deep_nest.cbor, which is 4,000 chains of 40 nested arrays,
that is 160,000 calls, and returning a 24-byte struct by value instead of filling one
moved that file from 1.25x to 1.68x against the C. Mean across the corpus went 1.04x to
1.11x. The struct is three words; returning it puts it through the return slot on every
level, where filling a caller's stack slot leaves it where the next call already wants it.
So the out-parameter stayed and this note exists instead. The shim's own
cbor_value_enter_container was always going to have that shape, since the C signature demands
it, so the inconsistency is confined to one internal function that the FFI boundary was
already forcing.
Recorded because "make it idiomatic" is the default advice for a port, and this is a place where taking it made the thing measurably worse. One measurement of a rejected variant, not a published headline.
15. cc appears under the fuzz crate, and nowhere else
Worth writing down before someone finds it and assumes the worst. fuzz/Cargo.toml depends
on libfuzzer-sys, which builds libFuzzer's own C++ runtime and therefore pulls cc into
that crate's build-dependencies. It is confined to the fuzzing binary, which is a test
harness and is never shipped.
The artifact under judgement is target/release/libtinycbor.a. cargo tree at the
workspace root lists only the four crates in this repository, with no build script and no
cc anywhere in it, and fuzz/ is deliberately not a workspace member. The C oracle the
fuzzer compares against is a separate executable driven over a pipe, per entry 5.
So: no C in the library, and the one place a C compiler runs at all is the tool that proves the library matches C.
16. cbor_value_advance skips by scanning, where upstream descends
Upstream walks a subtree recursively, decoding every item it passes into a CborValue
on the way. Then it throws all of that away: the only things that outlive the walk are
where the cursor ended up and whether anything was malformed. On a flat array of
integers that cost 106 instructions an item in this port and 117 in the C, where the
walk itself needs about ten.
So the buffer source has scan_subtree, which reads heads and adds lengths in one flat
loop with a small stack of how many items each open container still owes. Nesting is
not otherwise interesting when skipping: a container of N items is just N more items to
get through. It is the same traversal, without the bookkeeping nobody reads.
This is the one place the port stops being a transliteration, so two things keep it honest.
It can only be right or absent. The scan hands back to the recursive code the moment
it meets anything it does not want to reason about: a malformed head, a length
enter_container would reject, nesting past 64 levels, a break where one may not go. It
never reports an error itself. Every error the API can produce still comes from the
original code, on the original path, so the error taxonomy cannot drift.
It is checked against the C directly. tests/port/tst_advance_diff.c replays inputs
through cbor_value_advance and prints the error code, the final cursor offset and the
resulting type. Built against both archives and diffed, it agrees on all 3,980 inputs of
the fuzz, regression and benchmark corpora. make test runs the nine that ship.
Parsing went from 0.98x of the C to 0.36x, which is 2.8x faster, and every corpus file
improved. deep_nest, which had been the worst at 1.25x and resisted three earlier
rounds of tuning, came out at 0.32x.
Two things the scan cannot do, both by construction rather than by omission. A tag at the top of an advance stops on the tagged value rather than past it, because a tag prefixes an item instead of being one, and there is no enclosing loop to carry on into. And a reader source has no buffer to scan, so it keeps the recursive path entirely.
17. The JSON converter inherits an upstream bug, on purpose
cbor_value_to_json_advance does not validate UTF-8, although upstream's own header
comment says it does:
These functions also perform UTF-8 validation in CBOR text strings. If they encounter a sequence of bytes that is not permitted in UTF-8, they will return
CborErrorInvalidUtf8TextString. That includes encoding of surrogate points in UTF-8.
cbor_value_to_pretty_advance does exactly that. The JSON path does not, because text
strings go through escape_text_string(), which escapes what JSON requires escaping and
never decodes UTF-8. So 61 ff returns CborErrorInvalidUtf8TextString from one renderer
and CborNoError plus a JSON document that is not UTF-8 from the other. RFC 8259 §8.1
requires JSON to be UTF-8, so the successful call produced something a JSON parser will
refuse.
Filed as intel/tinycbor#331, against
9441b2ca, which is the current tip.
This port reproduces the behaviour exactly, and that is the decision. The whole claim
here is behavioural equivalence with a specific commit, checked by a differential fuzzer;
fixing a bug the reference still has would turn a passing check into a failing one and
make the equivalence claim false. The 1.4 million executions json_diff has run clean are
only meaningful because this port is bug-compatible.
If upstream takes the fix, this port follows it. Until then the divergence would be ours, not theirs.
Found by asking a different question than the fuzzer asks. The fuzzer compares this port against upstream, so anything both get wrong is invisible to it. Taking upstream's output for every successful conversion in the corpus and handing it to a strict JSON parser is a comparison against the specification instead, and 760 of roughly 27,000 conversions came back unparseable.
18. The tools are a second implementation, and it had drifted
cbordump and json2cbor are rewritten in safe Rust rather than being C over the
library. They have to be: cbor-ffi speaks in raw pointers, and a binary that consumed
that API would need unsafe in it, which is exactly what the budget is meant to keep in
one place. So tools/cbordump/src/pretty.rs is a second diagnostic-notation printer next
to crates/cbor-ffi/src/pretty.rs, and tools/cbordump/src/cbor.rs a second parser.
Nothing checked the second one. Upstream's Qt suite tests the library, and both differential fuzzers call the C ABI directly, so a tool could disagree with upstream on every input and the whole board would still read green. Running both tools against upstream's binaries on the 4,509-document fuzz corpus, over every combination of the four flags the JSON path recognises, found four bugs:
- an indefinite map with an odd number of items — a key with no value — was walked past rather than refused when it sat below the recursion limit, so the tool exited 0 on input upstream exits 1 on. That is the only one of the four that accepted something invalid rather than reporting the wrong error;
- a tag at the recursion limit prints a marker and leaves the item it tags unread, and upstream's container loop, driven by an item count that tags never decrement, renders that item on the next turn. Indexing the loop instead left the item as trailing bytes and reported garbage after the end of a document upstream renders in full;
- a break arriving where a map value is due, on a pair boundary, is upstream's
CborInvalidType: the wordinvalidandCborErrorUnknownType. This is the same arm the library's printer was missing, found by the fuzzer and fixed in an earlier round — the second copy never got it; - stepping off a tag head skips the preparse upstream does before it consults the recursion budget, so a document ending on a tag blamed nesting instead of saying it had run out of data.
The harness is tests/port/tools_diff.sh and it runs in make test. It needs upstream's
tools built, which are binaries rather than the vendored archive, so a machine without
them skips rather than fails.
The lesson is the one the fuzzing page already makes in a different key: a green board measures what is wired to it. Three of these four are error-path behaviour that no reasonable person would have found by reading, and the fourth was a validity bug sitting in a tool that had been green since the day it was written, because nothing had ever asked it a question.
19. Half-precision floats are done on the bit patterns, by hand
f16 is still unstable in Rust and the half crate is out of scope, so
cbor-core/src/half.rs converts binary16 both ways in 138 lines of shifts and masks.
Upstream does not have this problem in the same shape. Its decode_half is the RFC 7049
reference implementation, and its encode_half casts through (_Float16) where the
compiler supports one and falls back to the reference code where it does not. This port
has no such fallback to pick between: there is one implementation and it has to round the
way the hardware would.
That is the part worth stating. Decoding is easy — every binary16 value fits exactly in an
f32, so it cannot round at all, and the only care needed is renormalising subnormals and
keeping quiet NaNs quiet by shifting the payload rather than rebuilding it. Encoding is
where the bodies are: round-to-nearest-even, including the tie case, including values
that round up into a subnormal, and including values that round up into infinity. A naive
truncate-the-mantissa encoder passes an astonishing number of tests and is wrong on exactly
the inputs a fuzzer finds first.
core has no powi and no float formatting, which rules out the arithmetic shortcuts, so
everything is integer work on the raw bits. That turns out to be the right shape anyway: it
is branch-predictable, it has no floating-point environment to depend on, and it gives the
same answer on a machine with no FPU — which matters for a library whose audience is
microcontrollers.
encode_diff drives cbor_encode_float_as_half_float with arbitrary f32 bit patterns
against upstream, which is the check that the rounding actually matches rather than merely
looking like it should.
20. The parser does not allocate; one function does, and it uses malloc
Upstream's selling point is that parsing takes no heap. This port keeps that literally:
cbor-core is no_std + alloc, but nothing on the parse path calls into alloc.
Indefinite-length strings are the case that tempts you. A text string split into chunks
across the wire has no contiguous representation, so the obvious move is to concatenate it
into a Vec<u8> and hand back a slice. Upstream does not do that, and neither does this.
iterate_string_chunks walks the chunks twice with two different operations — Measure to
total the length, then CopyOut to fill the caller's buffer — and the caller owns the
memory both times. That is why cbor_value_copy_text_string takes a buffer and a
length-in-out parameter instead of returning something: the two-pass shape is the API, not
an implementation detail, and a port that allocated instead would silently change what
CborErrorOutOfMemory means to every existing caller.
There is exactly one allocation in the whole library, in _cbor_value_dup_string, and it
is malloc rather than Rust's allocator on purpose. The documented contract is that the
caller releases the result with free(). That names the allocator. Handing back a pointer
from Rust's global allocator would be a heap mismatch the compiler cannot see and the
caller cannot be blamed for — it would work today, because Rust's global allocator is
malloc on this target, and break silently the first time someone set
#[global_allocator]. Two extern "C" declarations are a smaller price. This is the only
reason alloc is a dependency at all.
21. The self-referential cursor is modelled as a raw pointer, not fought
CborValue holds a *const CborParser pointing at a parser the caller owns, and nothing in
the C API ties the two lifetimes together. A caller can free the parser and keep the value,
or copy the value into a container that outlives the parser, and the C compiler will not
say a word.
This is the shape Rust exists to prevent, and there is no way to express it safely without
changing the ABI — which is the one thing that cannot change, because the Qt tests read
these fields directly through static inline accessors. So the borrow checker is not
fought here, it is stepped around: the field stays a raw pointer, every dereference lives in
cbor-ffi behind a // SAFETY: line, and the invariant is named as the caller's to uphold
because in C it always was.
What made this tractable was pushing the decision out of the pointer and into the type
system one level up. A CborValue can be reading a flat buffer or calling back through a
caller-supplied operations table, and upstream tests a flag on every access to tell which.
Here the byte source is a type parameter instead, trait Source with Buffer and Reader
implementations, so the choice is made once at the call boundary and the compiler
monomorphises two specialised parsers out of one body — decision 13,
and the reason parsing ended up faster than the C rather than merely as fast.
The lifetime is still unenforced. Saying so plainly is better than a PhantomData that
implies a guarantee the ABI cannot make.
22. The unsafe budget: 80 blocks, all of them in the shim
cbor-core is #![forbid(unsafe_code)] — not deny, forbid, so no inner module can
opt back in. That crate is the entire CBOR implementation: the encoder, the parser's
logic, half-floats, formatting, validation. It compiles with no unsafe at all.
Every one of the 80 blocks is in cbor-ffi, and each carries a // SAFETY: line naming
the invariant the C caller has to uphold. They are almost all the same block: a C caller
handed us a pointer and we are about to read through it. That is what an ABI shim is —
the number is a measure of how many places C values cross into Rust, not of how much
risky code was written.
The count is published in the readme and it has grown over the weekend, from 61 to 80,
because the subtree scan and the JSON converter both added entry points. It is reported as
it stands rather than as it was when it looked better. For scale, uv ships 73 blocks and
Bun ships 13,044.
Two things the budget deliberately does not do. It does not count unsafe fn declarations
or extern "C" blocks, only unsafe { }, because those are the places something can
actually go wrong. And it does not chase the number down by wrapping several dereferences
in one block to make the total look smaller — that would trade a real property, one
invariant named per operation, for a better-looking figure.
23. Upstream's archive is committed, and the port tests link against it
bench/reference/libtinycbor-upstream.a is 76 KB of upstream compiled at the pinned commit,
checked into the repo. Four tests under tests/port/ are each built twice from one C source
file, once against that archive and once against ours, and the two transcripts are diffed.
The alternative was to build upstream during make test. That would have meant a fresh
clone could not run the differential tests without also cloning and building tinycbor, and
the whole point of those tests is that they run. So the comparison target ships with the
repo.
A static archive of C in a repository whose first claim is "no C" invites scrutiny, so here
is what it is not. It is not linked into libtinycbor.a, which is what a caller gets, and
not into cbordump or json2cbor. It is linked into build/*-c — test binaries whose only
job is to produce the transcript that ours is checked against. Deleting it would not change
a byte of the shipped library; it would only remove the evidence that the library agrees
with the thing it replaces.
make symbols and the ABI layout test read the other file in that directory,
symbols-upstream.txt, for the same reason: the reference has to be in the repo or the
check is not reproducible by a stranger.
24. The work is timestamped, and the timestamps are the proof
A port of a known library is exactly the kind of entry where somebody will wonder whether it was written before the window opened. The answer is in the history, and it does not need to be taken on trust.
The window opened 2026-07-31 18:00 UTC. The first commit is 18:24:04 UTC, twenty-four
minutes later, and it is an empty tree: git init happened inside the window, not before
it, and the first file arrives five minutes after that. The last commit is
2026-08-02 04:33:09 UTC.
git log --format='%h %ad %cd' --date=iso-strict # 100 commits, none before 18:24
Between those two the 100 commits fall across twelve distinct hours with gaps where a person was asleep, which is what real work looks like and what a single squashed drop does not. More usefully, nothing has been rewritten:
git log --format='%h %at %ct' | awk '$2!=$3' | wc -l # 0
Author date and committer date are equal on every commit. A rebase, a filter-branch, or a
backdated --date= all leave those two fields disagreeing, so a zero there rules out the
cheap ways of manufacturing a history after the fact. It is a weaker claim than a signed
timestamp and a stronger one than a tidy commit log, and it is the honest thing available.
The same reasoning is why fuzz/history.tsv records the commit each fuzz run ran against,
and why .port-mortem.toml carries kickoff_utc: a number that cannot be tied to a moment
is a number a reader has to believe.
25. The fuzz oracle falls back to the committed archive
fuzz/oracle/build.sh used to require a tinycbor checkout at ~/tinycbor-upstream and exit
if there was not one. That was fine on the machine this was written on and wrong everywhere
else: it meant the differential fuzzer, which is most of the evidence for behavioural
equivalence, could not be run by anyone who had only cloned this repository. They would have
had to take fuzz/log.txt on trust, and a log nobody else can regenerate is barely evidence
at all.
The oracle includes exactly two upstream headers, cbor.h and cborjson.h. Both are already
vendored in crates/cbor-ffi/include/, and upstream compiled at the pinned commit is already
committed as bench/reference/libtinycbor-upstream.a for the reasons in
23. So the pieces
were all here and the script simply was not looking for them. It does now, and prints which
source it used.
A checkout still takes precedence when one exists, because pointing TINYCBOR= at a
different upstream commit is the only way to fuzz against one, and that is worth keeping.
This closes the last target that needed something outside the repository. make, make test, make symbols, make lint, make bench and make fuzz all run from a fresh clone.
make test-tools still does not, because it compares against upstream's cbordump and
json2cbor binaries rather than a library, and vendoring built executables is a different
and worse proposition than vendoring a 76 KB archive. It skips with an explanation instead of
failing.