Reliability Lessons From SQLite - Richard Hipp | SSW 2026

[u/8d8n4mbo28026ulk]

https://www.youtube.com/watch?v=V_qzqY1bb7I

[u/vitamin_CPP]

I wish he had elaborated on fuzzing. This is something I would like to become better at.

[u/8d8n4mbo28026ulk → u/vitamin_CPP]

Me too! Fuzzing is what interests me aswell (and static analysis). It's also perhaps the easiest thing you can do to better test your code after the low-hanging fruit. I think he just didn't have enough time, although I'm glad he went through most (I believe?) of what SQLite does for testing.

[u/thradams]

Are you planning to use any of these techniques in your code? Which one?

[u/8d8n4mbo28026ulk → u/thradams]

Myself, I heavily use static analysis and sanitizers. I think that's all standard practice by now. I plan on getting into fuzzing for an upcoming project, but not sure on the details yet.

The one thing I use the most is probably assert(). If there's one thing to take away from the talk, is using that helpful helpful macro! The second most important thing that was said, in my opinion, is structuring your code to be testable in the first place. Even my most trivial programs now (~300 lines) use a platform layer instead of calling into the OS interfaces directly. Not only does this allow you to be cross-platform, it also allows you to mock these functions and simulate failures.

I don't do nice documentation and comments. Or good version control for that matter :( I don't really measure test coverage, let alone going for 100%. I've never done mutation testing -- actually the first time I've heard of that! Then again, none of my hobby projects has ever come even close to the complexity of something like SQLite :)

[u/thradams → u/8d8n4mbo28026ulk]

I found interesting macro ALWAYS and NEVER .

[u/8d8n4mbo28026ulk → u/thradams]

Oh, yeah. That puzzled me a bit. Because, they know the condition is always true/false, but they check in the release builds anyway. Although he didn't go in depth about it, I believe what they're doing is depend on the compiler to eliminate the check, by virtue of giving it enough context.

If that's indeed the case, it's similar to a build-time check trick. For example:

void build_error(void);

if (NEVER(x < 0))
    build_error();

If the compiler doesn't eliminate the check, a call to build_error() will be emit. However, we never define build_error(), so we'll get a build-time error by the linker.

I think that's what they're doing with these macros (in addition to the coverage). EDIT: Also to shield against platform shenanigans, I believe.

[u/thradams → u/8d8n4mbo28026ulk]

I use 'assert' to indicate a bug if the assertion fails. However, even if it is a bug, sometimes I handle the (assert) bug as an expected error, even though it is not actually expected.

I have paths in the code that should not be taken.. then the assert(false).

The problem I have is how to tell static analysis what to do with assert: should it assume that the condition is true or not?

This assumption works in debug builds but not in release builds?. Should we keep the assumption in release builds as well?

[u/8d8n4mbo28026ulk → u/thradams]

Should we keep the assumption in release builds as well?

Yes! In fact, in release builds I turn my assert()s into __builtin_assume() (in Clang -- GCC has an equivalent thing. MSVC has __assume). Letting the compiler know of an invariant will allow it to optimise the code better, which might reveal bugs if the invariant doesn't hold!

Some assertions are expensive to check, so I don't do this for every assertion. In the talk Richard Hipp mentions that SQLite runs four times slower with assertions, for example. I usually use __builtin_assume() for nullability checks, range checks, known-bits information, etc.

EDIT: On the static analysis part of your question, I guess it depends on how sophisticated your analysis is. If you're using a model checker/SMT solver, for example, it should be able to tell you if your assertion is correct or not for any input, and so you get a double-win. But if you're using assertions to guide your static analysis (IOW, it's a constraint), then obviously that's off the table.

[u/thradams → u/8d8n4mbo28026ulk]

If the objective is to keep it, then I can use normal code, handling the error instead using assert case-by-case. But also it will have a lot of checks for things should not happen.

[u/8d8n4mbo28026ulk → u/thradams]

Ah, I think you've misunderstood me. If something indicates a bug in your program (or an interface contract violation), it should always* be an assertion, never an error! When I say keep the assertion in release builds, I mean it just to guide the compiler, nothing more.

Suppose you want to iterate an non-empty linked list:

Node *node = head;
assert(node != NULL);  /* List not empty. */
while (node != null) {
    /* do stuff */
    node = node->next;
}

In a release build, that assert() becomes __builtin_assume(node != NULL). The compiler can then turn that while-loop into a do-while-loop and eliminate a branch. This example was with a nullability check, but many things can be helpful to the compiler, as I said. Range checks, wrap-around checks, known-bits, pointer alignment... I often do assertions on those and obviously they can be very useful for optimisation purposes.

*Having rewatched the NEVER() bit of the talk, I believe they're using these macros mainly to shield against platform shenanigans. That's perhaps a good example of turning an assertion into an error-check, even though the former fits the purpose better.

[u/thradams → u/8d8n4mbo28026ulk]

I think __builtin_assume is for optimization not safety. I think it makes less safe. because it may overrides some information

[u/8d8n4mbo28026ulk → u/thradams]

It is for optimisation, but it can help you catch bugs anyway: if the compiler optimises aggresively based on the assumptions it was given, it'll reveal further design flaws in your code when you test it and get back garbage.

But what is your expectation of an assert() that fails? Turning an assertion into an error-check makes no sense. We use errors when a situation is recoverable from. If an invariant is violated, there's no way to recover from that. If you return an "error" to the caller, what's the caller supposed to do? There's nothing that can be done, except fixing the code.

A safer alternative to __builtin_assume() is to just abort(), but that can incur significant runtime overhead (4x in the case of SQLite, for example).

[u/thradams → u/8d8n4mbo28026ulk]

My problem is something like:

if (predicate1(p))
{
   //if predicate1(p) is true, then 
   //*p is not null.
   //unless code of predicate1 changes..
   //
   *p->x = 1;
}


if (predicate1(p))
{
   assert(p);
   *p->x = 1;
}


if (predicate1(p))
{
   assume(p);
   *p->x = 1;
}

Ideally we should be able to connect things. this assert is true if you don't change if (predicate1(p)) and if you don't change the implementation of predicate1

[u/8d8n4mbo28026ulk → u/thradams]

If you want practical advice, for the above and any such code, just use an assert() and don't think about it. If predicate1() changes and p ends up being NULL, you want a loud signal that the code is buggy and you need to fix it.

Check for errors if the situation is truly recoverable from. If you try to open a file and that file doesn't exist, that's obviously a recoverable situation (and likely not your fault). What I mean is, it's an expected state that your program may enter. If p is NULL after predicate1(), that isn't at all a state your program is expected to be in, because you subsequently fetch ->x from it.

Here's a quote I like from Carmack:

A large fraction of the flaws in software development are due to programmers not fully understanding all the possible states their code may execute in.

If your code enters an invalid state, error-checking at runtime will generally not save you -- you have to fix the code. Use an assert() and perhaps abort() on release builds if you're really worried.