![[Linux]](http://worker.jart.workers.dev/redbean/linux.png "Linux"){kind=small}
![[Windows]](http://worker.jart.workers.dev/redbean/windows10.png "Windows"){kind=small}
![[MacOS]](http://worker.jart.workers.dev/redbean/macos.png "MacOS"){kind=small}
![[FreeBSD]](http://worker.jart.workers.dev/redbean/freebsd64.png "FreeBSD"){kind=small}
![[OpenBSD]](http://worker.jart.workers.dev/redbean/openbsd.png "OpenBSD"){kind=small}
![[NetBSD]](http://worker.jart.workers.dev/redbean/netbsd2.png "NetBSD"){kind=small}
![[red stuffed lobster logo]](http://worker.jart.workers.dev/redbean2/redbean.png)
redbean is an open source webserver in a single-file that runs natively on six OSes for both AMD64 and ARM64. Basic idea is if you want to build a web app that runs anywhere, then you download the redbean.com file, put your .html and .lua files inside it using the zip command, and you've got a hermetic app you deploy and share.
redbean embeds Lua, SQLite, and MbedTLS into a fork() driven application server that benchmarks at 5.3 million qps on Threadripper. It's got a live bestline REPL with code completion and a UNIX module too, that lets you directly use the Cosmopolitan Libc system call interface. redbean furthermore provides sandboxing and system call tracing for security. This makes redbean a great fit for when you want to build an app that's vertically integrated into a single tiny file that runs on nearly all PCs and servers.
|
documentation |
modules |
reference community |
redbean v3.0.0 was released on 2024-08-17.
redbean-3.0.0.com
5.5m - PE+ELF+MachO+ZIP+SH for AMD64 and ARM64 (source code)
382f1288bb96ace4bab5145e7df236846c33cc4f1be69233710682a9e71e7467
redbean-demo-3.0.0.com
5.5m - same as redbean but includes example code
cf7b6dcef357bf1d3a08e373ffd7fa41b0f343d0b66ddc22776e0d273082f727
# build once run anywhere using linux
git clone https://github.com/jart/cosmopolitan
cd cosmopolitan
make -j8 o//tool/net/redbean
o//tool/net/redbean -vv
On cosmo.zip you'll find other command line tools, built as APE binaries, that are essential when developing software with redbean.
redbean is designed to not need to be installed. Since there's no dependencies, all you need to do is download the binary and run it.
curl https://redbean.dev/redbean-demo-latest.com >redbean-demo.com chmod +x redbean-demo.com ./redbean-demo.com -v
That'll start up a command line web server with a REPL. If you're on a desktop, it'll also launch a tab in your browser, visiting your redbean web server. If you go to http://localhost:8080/ then you should be able to see a listing of the content, with many demo scripts.
Having trouble? Here are some more specific instructions for various platforms, in case the above commands don't work.
On macOS with Apple Silicon you need to have Xcode Command Line Tools installed for redbean to be able to bootstrap itself.
If you use zsh and have trouble running redbean, try saying sh -c
./redbean-demo.com. This is due to a bug that was fixed in zsh 5.9+.
The same is the case for Python subprocess, old versions of
Fish, etc.
If you get the MacOS error "... cannot be opened because the developer cannot be verified" then:
sudo
spctl --master-disable; [llama launch command]; sudo spctl
--master-enable. This is because --master-disable
disables _all_ checking, so you need to turn it back on after quitting
llama.
On some Linux systems, you might get errors relating to `run-detectors` or WINE. This is due to `binfmt_misc` registrations. You can fix that by adding an additional registration for the APE file format redbean uses:
sudo wget -O /usr/bin/ape https://cosmo.zip/pub/cosmos/bin/ape-$(uname -m).elf sudo chmod +x /usr/bin/ape sudo sh -c "echo ':APE:M::MZqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register" sudo sh -c "echo ':APE-jart:M::jartsr::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register"
Generally speaking with Windows, you just need to make sure the APE binary has a .exe or .com extension. To make things simple for you:
curl -o zip.exe https://cosmo.zip/pub/cosmos/bin/zip curl -o redbean-demo.com https://redbean.dev/redbean-demo-latest.com .\redbean-demo.com
On WSL, there are many possible gotchas. One thing that helps solve them is this:
[Unit] Description=cosmopolitan APE binfmt service After=wsl-binfmt.service [Service] Type=oneshot ExecStart=/bin/sh -c "echo ':APE:M::MZqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register" [Install] WantedBy=multi-user.target
Put that in /etc/systemd/system/cosmo-binfmt.service. You
also need to
download ape-x86_64.elf
and copy it to /usr/bin/ape with chmod +x.
Then run sudo systemctl enable cosmo-binfmt.
Another thing that's helped WSL users who experience issues, is to disable the WIN32 interop feature:
sudo sh -c "echo -1 > /proc/sys/fs/binfmt_misc/WSLInterop"
In the instance of getting a Permission Denied on disabling interop
through CLI, it can be permanently disabled by adding the following in
/etc/wsl.conf.
[interop] enabled=false
Here's basic overview of how you'd go about using redbean alongside InfoZIP which is a free tool that comes included with most UNIX systems.
echo '<b>hello</b>' >hello.html zip redbean.com hello.html # in first terminal $ ./redbean.com -vv I2022-04-29T07:17:28+000767:redbean] (srvr) listen http://127.0.0.1:8080 >: waiting for command... # in second terminal $ curl http://127.0.0.1:8080/hello.html <b>hello</b> $ curl -k https://127.0.0.1:8080/hello.html <b>hello</b> # add some stuff to the zip executable $ echo 'Write("<b>hello</b>")' >hello.lua $ zip redbean.com hello.lua # perform a naked http request $ printf 'GET /hello.lua\n\n' | nc 127.0.0.1 8080 <b>hello</b>
One GUI-like feature redbean itself offers is the ability to
LaunchBrowser() at startup,
which can be called from your
your /.init.lua file.
You have greater power to customize your redbean build if you build from source. For example:
git clone https://github.com/jart/cosmopolitan && cd cosmopolitan make -j8 MODE=tiny o/tiny/tool/net ls o/tiny/tool/net/redbean*.com o/tiny/tool/net/redbean.com -vv make -j8 MODE=tiny o/tiny/tool/net o/tiny/tool/net/redbean.com -vv make -j8 MODE=tinylinux o/tinylinux/tool/net o/tinylinux/tool/net/redbean-original.com -vv
Some of the supported build modes are: MODE= (default), MODE=opt, rel, dbg, tiny, asan, optlinux, tinylinux, tinylinuxbsd, and tinysysv.
You can launch redbean in your terminal in the most verbose way possible by doing the following:
./redbean.com -vvmbag # starts server verbosely open http://127.0.0.1:8080/ # shows zip listing page CTRL-C # 1x: graceful shutdown CTRL-C # 2x: forceful shutdown
Assets can be added to the zip archive as follows:
zip redbean.com index.html # adds file zip -r redbean.com mirrored-website # adds directory echo comment | zip -c redbean.com foo.html # adds file w/ comment
By default, anything you add to the archive gets compressed. Sometimes you don't want that to happen. A good example is video files. The web browser will want to send HTTP range requests to seek in the video, in which case redbean requires that the asset be uncompressed.
zip -0 redbean.com video.mp4 # adds file without compression
You can run redbean interactively in your terminal as follows:
redbean.com -vv CTRL-C # 1x: graceful shutdown CTRL-C # 2x: forceful shutdown
The index.lua and index.html names are special
since they're used to automatically figure out how to serve directories.
Such files can appear in any directory, however the root directory is
special. The default action for / is to show a listing page
showing the contents of your zip central directory.
The listing page only applies to the root directory. However the default index page applies to subdirectories too. In order for it to work, there needs to be an empty directory entry in the zip. That should already be the default practice of your zip editor.
redbean normalizes the trailing slash for you automatically:
$ printf 'GET /a.example HTTP/1.0\n\n' | nc 127.0.0.1 8080
HTTP/1.0 307 Temporary Redirect
Location: /a.example/
Virtual hosting is accomplished this way too. The Host is simply prepended to the path, and if it doesn't exist, it gets removed.
$ printf 'GET / HTTP/1.1\nHost:a.example\n\n' | nc 127.0.0.1 8080
HTTP/1.1 200 OK
Link: <http://127.0.0.1/a.example/index.html>; rel=\"canonical\"
If you mirror a lot of websites within your redbean then you can actually tell your browser that redbean is your proxy server, in which redbean will act as your private version of the Internet.
wget \ --mirror \ --convert-links \ --adjust-extension \ --page-requisites \ --no-parent \ --no-if-modified-since \ http://a.example/index.html zip -r redbean.com a.example/ # default page for directory printf 'GET http://a.example HTTP/1.0\n\n' | nc 127.0.0.1 8080 HTTP/1.0 200 OK Link: <http://127.0.0.1/a.example/index.html>; rel="canonical"
If you use a reverse proxy, then redbean recognizes the following provided that the proxy forwards requests over the local network:
X-Forwarded-For: 203.0.113.42:31337 X-Forwarded-Host: foo.example:80
There's a text/plain statistics page called /statusz that makes it easy to track and monitor the health of your redbean:
printf 'GET /statusz\n\n' | nc 127.0.0.1 8080
redbean will display an error page using the /redbean.png logo by default, embedded as a bas64 data uri. You can override the custom page for various errors by adding files to the zip root.
zip redbean.com 404.html # custom not found page
| -h or -? | help |
|---|---|
| -d | daemonize |
| -u | uniprocess |
| -z | print port |
| -m | log messages |
| -i | interpreter mode |
| -b | log message bodies |
| -a | log resource usage |
| -g | log handler latency |
| -E | show crash reports to public ips |
| -j | enable ssl client verify |
| -k | disable ssl fetch verify |
| -Z | log worker system calls |
| -f | log worker function calls |
| -B | only use stronger cryptography |
| -X | disable ssl server and client support |
| -* | permit self-modification of executable |
| -J | disable non-ssl server and client support |
| -% | hasten startup by not generating an rsa key |
| -s | increase silence [repeatable] |
| -v | increase verbosity [repeatable] |
| -V | increase ssl verbosity [repeatable] |
| -S | increase pledge sandboxing [repeatable] |
| -e CODE | eval Lua code in arg [repeatable] |
| -F PATH | eval Lua code in file [repeatable] |
| -H K:V | sets http header globally [repeatable] |
| -D DIR | overlay assets in local directory [repeatable] |
| -r /X=/Y | redirect X to Y [repeatable] |
| -R /X=/Y | rewrites X to Y [repeatable] |
| -K PATH | tls private key path [repeatable] |
| -C PATH | tls certificate(s) path [repeatable] |
| -A PATH | add assets with path (recursive) [repeatable] |
| -M INT | tunes max message payload size [def. 65536] |
| -t INT | timeout ms or keepalive sec if <0 [def. 60000] |
| -p PORT | listen port [def. 8080; repeatable] |
| -l ADDR | listen addr [def. 0.0.0.0; repeatable] |
| -c SEC | configures static cache-control |
| -W TTY | use tty path to monitor memory pages |
| -L PATH | log file location |
| -P PATH | pid file location |
| -U INT | daemon set user id |
| -G INT | daemon set group id |
| -w PATH | launch browser on startup |
| --strace | enables system call tracing (see also -Z) |
| --ftrace | enables function call tracing (see also -f) |
| ctrl-d | exit |
|---|---|
| ctrl-c ctrl-c | exit |
| ctrl-e | end |
| ctrl-a | start |
| ctrl-b | back |
| ctrl-f | forward |
| ctrl-l | clear |
| ctrl-h | backspace |
| ctrl-d | delete |
| ctrl-n | next history |
| ctrl-p | previous history |
| ctrl-r | search history |
| ctrl-g | cancel search |
| alt-< | beginning of history |
| alt-> | end of history |
| alt-f | forward word |
| alt-b | backward word |
| ctrl-right | forward word |
| ctrl-left | backward word |
| ctrl-k | kill line forwards |
| ctrl-u | kill line backwards |
| alt-h | kill word backwards |
| ctrl-w | kill word backwards |
| ctrl-alt-h | kill word backwards |
| alt-d | kill word forwards |
| ctrl-y | yank |
| alt-y | rotate kill ring and yank again |
| ctrl-t | transpose |
| alt-t | transpose word |
| alt-u | uppercase word |
| alt-l | lowercase word |
| alt-c | capitalize word |
| ctrl-\ | quit process |
| ctrl-s | pause output |
| ctrl-q | unpause output (if paused) |
| ctrl-q | escaped insert |
| ctrl-alt-f | forward expr |
| ctrl-alt-b | backward expr |
| alt-right | forward expr |
| alt-left | backward expr |
| alt-shift-b | barf expr |
| alt-shift-s | slurp expr |
| ctrl-space | set mark |
| ctrl-x ctrl-x | goto mark |
| ctrl-z | suspend process |
| alt-\ | squeeze adjacent whitespace |
| protip | remap caps lock to ctrl |
Any files with the extension .lua will be dynamically
served by redbean. Here's the simplest possible example:
Write('<b>Hello World</b>')
The Lua Server Page above should be able to perform at 700,000 responses
per second on a Core i9, without any sort of caching. If you want a Lua
handler that can do 1,000,000 responses per second, then try adding the
following global handler to your /.init.lua file:
function OnHttpRequest() Write('<b>Hello World</b>') end
Here's an example of a more typical workflow for Lua Server Pages using the redbean API:
SetStatus(200) SetHeader('Content-Type', 'text/plain; charset=utf-8') Write('<p>Hello ') Write(EscapeHtml(GetParam('name')))
We didn't need the first two lines in the previous example, because they're implied by redbean automatically if you don't set them. Responses are also buffered until the script finishes executing. That enables redbean to make HTTP as easy as possible. In the future, API capabilities will be expanded to make possible things like websockets.
redbean embeds the
Lua standard library.
You can use packages such as
io
to persist and share state across requests and connections, as well as
the StoreAsset function, and
the lsqlite3 module.
Your Lua interpreter begins its life in the main process at startup in
the .init.lua, which is likely where
you'll want to perform all your expensive one-time operations like
importing modules. Then, as requests roll in, isolated processes are
cloned from the blueprint you created.
Your redbean displays a Read-Eval-Print-Loop that lets you modify the state of the main server process while your server is running. Any changes will propagate into forked clients.
Your REPL is displayed only when redbean is run as a non-daemon in a Unix terminal or the Windows 10 command prompt or PowerShell. Since the REPL is a Lua REPL it's not included in a redbean-static builds.
redbean uses the same keyboard shortcuts as GNU Readline and Emacs. Some of its keyboard commands (listed in a previous section) were inspired by Paredit.
A history of your commands is saved to ~/.redbean_history.
If you love the redbean repl and want to use it as your language
interpreter then you can pass the -i flag to put redbean
into interpreter mode.
redbean.com -i binarytrees.lua 15
When the -i flag is passed (for interpreter mode),
redbean won't start a web server and will instead functions like
the lua command. The first command line argument becomes
the script you want to run. If you don't supply a script, then the
repl without a web server is displayed. This is useful for testing
since redbean extensions and modules for the Lua language, are still
made available. You can also write redbean scripts with shebang lines:
#!/usr/bin/redbean -i print('hello world')
However UNIX operating systems usually require that interpreters be encoded in its preferred executable format. You can assimilate your redbean into the local format using the following commands:
$ file redbean.com redbean.com: DOS/MBR boot sector $ curl -o assimilate https://cosmo.zip/pub/cosmos/bin/assimilate $ chmod +x assimilate $ ./assimilate ./redbean.com $ file redbean.com redbean.com: ELF 64-bit LSB executable $ sudo cp redbean.com /usr/bin/redbean
By following the above steps, redbean can be installed systemwide for multiple user accounts. It's also possible to chmod the binary to have setuid privileges. Please note that, if you do this, the UNIX section provides further details on APIs like unix.setuid that will help you remove root privileges from the process in the appropriate manner.
We've made some enhancements to the Lua language that should make it more comfortable for C/C++ and Python developers. Some of these
"hello %s" % {"world"} instead of
string.format("hello %s", "world").
"hi" * 2 instead of
string.rep("hi", 2).
0644 == 420 is the case in redbean, whereas in upstream Lua
0644 == 644 would be the case.
0b1010 == 10 is the case in redbean, whereas in upstream Lua
0b1010 would result in an error.
"\e" is the same as "\x1b".
/index.lua or /index.html file defined.
ProgramFOO() functions below. The init module
load happens after redbean's arguments and zip assets have been
parsed, but before calling functions
like socket()
and fork(). Note that this path is a hidden
file so that it can't be unintentionally run by the network client.
package.path
to /zip/.lua/?.lua;/zip/.lua/?/init.lua by default.
Cosmopolitan Libc lets system calls like open read from
the ZIP structure, if the filename is prefixed
with /zip/. So this works like magic.
/ listing page icon,
embedded as a base64 URI.
TZ environment variable controls which one of these
files is used by functions such as unix.localtime().
There's one argument per line. Trailing newline is ignored. If the
special argument ... is not encountered, then the
replacement will only happen if no CLI args are specified. If
the special argument ... is encountered, then
it'll be replaced with whatever CLI args were specified by the user.
For example, you might want to use redbean.com in interpreter mode, where your script file is inside the zip. Then, if your redbean is run, what you want is to have the default behavior be running your script. In that case, you might:
$ cat <<'EOF' >.args
-i
/zip/hello.lua
EOF
$ cat <<'EOF' >hello.lua
print("hello world")
EOF
$ zip redbean.com .args hello.lua
$ ./redbean.com
hello world
Please note that if you ran:
$ ./redbean.com -vv
Then the default mode of redbean will kick back in. To prevent
that from happening, simply add the magic arg ... to the end
of your .args file.
For example, if you launch your redbean as follows:
redbean -v arg1 arg2
Then your /.init.lua file will have the arg
array like:
arg[-1] = '/usr/bin/redbean' arg[ 0] = '/zip/.init.lua' arg[ 1] = 'arg1' arg[ 2] = 'arg2'
If you launch redbean in interpreter mode (rather than web server) mode, then an invocation like this:
./redbean.com -i script.lua arg1 arg2
Would have an arg array like this:
arg[-1] = './redbean.com' arg[ 0] = 'script.lua' arg[ 1] = 'arg1' arg[ 2] = 'arg2'
/.init.lua then redbean will
call it at the earliest possible moment to hand over control for all
messages (with the exception of OPTIONS *). See
functions like Route which asks
redbean to do its default thing from the handler.
true then redbean will close the connection without
calling fork.
SetHeader('Connection','Close').
This won't be called in uniprocess mode.
SO_REUSEPORT, for
example. If it returns true, redbean will not listen to
that ip/port.
reason is optional since redbean can fill in the
appropriate text for well-known magic numbers,
e.g. 200, 404, etc. This method will reset the
response and is therefore mutually exclusive
with ServeAsset and other Serve*
functions. If a status setting function isn't called, then the default
behavior is to send 200 OK.
name is
case-insensitive and restricted to non-space
ASCII. value is a UTF-8 string that must be encodable
as ISO-8859-1. Leading and trailing whitespace is trimmed
automatically. Overlong characters are canonicalized. C0 and C1
control codes are forbidden, with the exception of tab. This
function automatically
calls SetStatus(200, "OK") if
a status has not yet been set. As SetStatus and Serve* functions
reset the response, SetHeader needs to be called after SetStatus and
Serve* functions are called. The header buffer is independent of the
payload buffer. Neither is written to the wire until the Lua Server
Page has finished executing. This function disallows the setting of
certain headers such as and Content-Range which are
abstracted by the transport layer. In such cases, consider
calling ServeAsset.
__Host- and __Secure- prefixes are supported
and may set or overwrite some of the options (for example, specifying
__Host- prefix sets the Secure option to true, sets the
path to "/", and removes the Domain option). The following options can
be used (their lowercase equivalents are supported as well):
Expires sets the maximum lifetime of the cookie as
an HTTP-date timestamp. Can be specified as a Date in the RFC1123
(string) format or as a UNIX timestamp (number of seconds).
MaxAge sets number of seconds until the cookie
expires. A zero or negative number will expire the cookie
immediately. If both Expires and MaxAge are set, MaxAge has
precedence.
Domain sets the host to which the cookie will be
sent.
Path sets the path that must be present in the
request URL, or the client will not send the Cookie header.
Secure (bool) requests the cookie to be only send to
the server when a request is made with the https: scheme.
HttpOnly (bool) forbids JavaScript from accessing
the cookie.
SameSite (Strict, Lax, or None) controls whether a
cookie is sent with cross-origin requests, providing some
protection against cross-site request forgery attacks.
name is
handled in a case-sensitive manner. This function checks Request-URL
parameters first. Then it
checks application/x-www-form-urlencoded from the
message body, if it exists, which is common for HTML forms
sending POST requests. If a parameter is supplied
matching name that has no value, e.g. foo
in ?foo&bar=value, then the returned value will
be nil, whereas for ?foo=&bar=value it
would be "". To differentiate between no-equal and
absent, use the HasParam
function. The returned value is decoded from ISO-8859-1 (only in the
case of Request-URL) and we assume that percent-encoded characters
were supplied by the client as UTF-8 sequences, which are returned
exactly as the client supplied them, and may therefore may contain
overlong sequences, control codes, NUL characters, and
even numbers which have been banned by the IETF. It is the
responsibility of the caller to impose further restrictions on
validity, if they're desired.
&><"' which
become &><"'.
This function is charset agnostic and will not canonicalize overlong
encodings. It is assumed that a UTF-8 string will be supplied.
See escapehtml.c.
EscapePath if needed, as
it's not escaped automatically. This function may be called from
your /.init.lua.
data: URIs that do things like embed a PNG file in
a web page.
See encodebase64.c.
This is a generally permissive parser, in the sense that like v8, it permits scalars as top-level values. Therefore we must note that this API can be thought of as special, in the sense
val = assert(DecodeJson(str))
will usually do the right thing, except in cases where false or null are the top-level value. In those cases, it's needed to check the second value too in order to discern from error
val, err = DecodeJson(str) if not val then if err then print('bad json', err) elseif val == nil then print('val is null') elseif val == false then print('val is false') end end
This parser supports 64-bit signed integers. If an overflow
happens, then the integer is silently coerced to double, as
consistent with v8. If a double overflows into Infinity, we
coerce it to null since that's what v8 does, and the same
goes for underflows which, like v8, are coerced to 0.0.
When objects are parsed, your Lua object can't preserve the original ordering of fields. As such, they'll be sorted by EncodeJson() and may not round-trip with original intent
This parser has perfect conformance with JSONTestSuite.
This parser validates utf-8 and utf-16.
Since Lua tables are both hashmaps and arrays, we use a simple fast
algorithm for telling the two apart. Tables with non-zero length (as
reported by #) are encoded as arrays, and any non-array
elements are ignored. For example:
>: EncodeJson({2})
"[2]"
>: EncodeJson({[1]=2, ["hi"]=1})
"[2]"
If there are holes in your array, then the serialized array will exclude everything after the first hole. If the beginning of your array is a hole, then an error is returned.
>: EncodeJson({[1]=1, [3]=3})
"[1]"
>: EncodeJson({[2]=1, [3]=3})
"[]"
>: EncodeJson({[2]=1, [3]=3})
nil "json objects must only use string keys"
If the raw length of a table is reported as zero, then we
check for the magic element [0]=false. If it's present, then
your table will be serialized as empty array []. An entry is
inserted by DecodeJson() automatically, only when encountering
empty arrays, and it's necessary in order to make empty arrays
round-trip. If raw length is zero and [0]=false is absent,
then your table will be serialized as an iterated object.
>: EncodeJson({})
"{}"
>: EncodeJson({[0]=false})
"[]"
>: EncodeJson({["hi"]=1})
"{\"hi\":1}"
>: EncodeJson({["hi"]=1, [0]=false})
"[]"
>: EncodeJson({["hi"]=1, [7]=false})
nil "json objects must only use string keys"
The following options may be used:
useoutput: (bool=false) encodes the result directly to
the output buffer and returns nil value. This option is
ignored if used outside of request handling code.
sorted: (bool=true) Lua uses hash tables so the order of
object keys is lost in a Lua table. So, by default, we
use strcmp to impose a deterministic output order. If you
don't care about ordering then setting sorted=false
should yield a performance boost in serialization.
pretty: (bool=false) Setting this option to true will
cause tables with more than one entry to be formatted across multiple
lines for readability.
indent: (str=" ") This option controls the
indentation of pretty formatting. This field is ignored
if pretty isn't true.
maxdepth: (int=64) This option controls the maximum
amount of recursion the serializer is allowed to perform. The max is
32767. You might not be able to set it that high if there isn't enough
C stack memory. Your serializer checks for this and will return an
error rather than crashing.
This function will return an error if:
value is cyclic
value has depth greater than 64
value contains functions, user data, or threads
value is table that blends string / non-string keys
We assume strings in value contain UTF-8. This serializer
currently does not produce UTF-8 output. The output format is
right now ASCII. Your UTF-8 data will be safely transcoded to
\uXXXX sequences which are UTF-16. Overlong encodings in your
input strings will be canonicalized rather than validated.
NaNs are serialized as null and Infinities are null which
is consistent with the v8 behavior.
Since Lua uses tables as both hashmaps and arrays, tables will only be serialized as an array with determinate order, if it's an array in the strictest possible sense.
In all other cases, your table will be serialized as an object which is iterated and displayed as a list of (possibly) sorted entries that have equal signs.
>: EncodeLua({3, 2})
"{3, 2}"
>: EncodeLua({[1]=3, [2]=3})
"{3, 2}"
>: EncodeLua({[1]=3, [3]=3})
"{[1]=3, [3]=3}"
>: EncodeLua({["hi"]=1, [1]=2})
"{[1]=2, hi=1}"
The following options may be used:
useoutput: (bool=false) encodes the result directly to
the output buffer and returns nil value. This option is
ignored if used outside of request handling code.
sorted: (bool=true) Lua uses hash tables so the order of
object keys is lost in a Lua table. So, by default, we
use strcmp to impose a deterministic output order. If you
don't care about ordering then setting sorted=false
should yield a performance boost in serialization.
pretty: (bool=false) Setting this option to true will
cause tables with more than one entry to be formatted across multiple
lines for readability.
indent: (str=" ") This option controls the
indentation of pretty formatting. This field is ignored
if pretty isn't true.
maxdepth: (int=64) This option controls the maximum
amount of recursion the serializer is allowed to perform. The max is
32767. You might not be able to set it that high if there isn't enough
C stack memory. Your serializer checks for this and will return an
error rather than crashing.
If a user data object has a __repr or __tostring meta
method, then that'll be used to encode the Lua code.
This serializer is designed primarily to describe data. For example, it's used by the REPL where we need to be able to ignore errors when displaying data structures, since showing most things imperfectly is better than crashing. Therefore this isn't the kind of serializer you'd want to use to persist data in prod. Try using the JSON serializer for that purpose.
Non-encodable value types (e.g. threads, functions) will be represented as a string literal with the type name and pointer address. The string description is of an unspecified format that could most likely change. This encoder detects cyclic tables; however instead of failing, it embeds a string of unspecified layout describing the cycle.
Integer literals are encoded as decimal. However if the int64
number is ≥256 and has a population count of 1 then we switch
to representating the number in hexadecimal, for readability.
Hex numbers have leading zeroes added in order to visualize
whether the number fits in a uint16, uint32, or int64. Also
some numbers can only be encoded expressionally. For example,
NaNs are serialized as 0/0, and Infinity is math.huge.
>: 7000 7000 >: 0x100 0x0100 >: 0x10000 0x00010000 >: 0x100000000 0x0000000100000000 >: 0/0 0/0 >: 1.5e+9999 math.huge >: -9223372036854775807 - 1 -9223372036854775807 - 1
The only failure return condition currently implemented is when C runs out of heap memory.
#fragment. The allowed characters
are -/?.~_@:!$&'()*+,;=0-9A-Za-z and everything
else gets %XX encoded. Please note
that '& can still break HTML and
that '() can still break CSS URLs. This function is
charset agnostic and will not canonicalize overlong encodings. It is
assumed that a UTF-8 string will be supplied.
See kescapefragment.S.
\uxxxx sequences for all non-ASCII
sequences. HTML entities are also encoded, so the output doesn't
need EscapeHtml. This
function assumes UTF-8 input. Overlong encodings are canonicalized.
Invalid input sequences are assumed to be ISO-8859-1. The output is
UTF-16 since that's what JavaScript uses. For example, some
individual codepoints such as emoji characters will encode as
multiple \uxxxx sequences. Ints that are impossible to
encode as UTF-16 are substituted with the \xFFFD
replacement character.
See escapejsstringliteral.c.
-.*_0-9A-Za-z and everything else
gets %XX encoded. This function is charset agnostic and
will not canonicalize overlong encodings. It is assumed that a UTF-8
string will be supplied.
See kescapeparam.S.
EscapeSegment except
slash is allowed. The allowed characters
are -.~_@:!$&'()*+,;=0-9A-Za-z/ and everything else
gets %XX encoded. Please note that '&
can still break HTML, so the output may
need EscapeHtml too. Also
note that '() can still break CSS URLs. This function
is charset agnostic and will not canonicalize overlong encodings. It
is assumed that a UTF-8 string will be supplied.
See kescapepath.S.
EscapePath except slash
isn't allowed. The allowed characters
are -.~_@:!$&'()*+,;=0-9A-Za-z and everything else
gets %XX encoded. Please note that '&
can still break HTML, so the output may
need EscapeHtml too. Also
note that '() can still break CSS URLs. This function
is charset agnostic and will not canonicalize overlong encodings. It
is assumed that a UTF-8 string will be supplied.
See kescapesegment.S.
method (default: "GET"): sets the
method to be used for the request. The specified method is
converted to uppercase.
body (default: ""): sets the body value to be
sent.
headers sets headers for the request using the
key/value pairs from this table. Only string keys are used and
all the values are converted to strings.
followredirect (default: true):
forces temporary and permanent redirects to be followed. This
behavior can be disabled by passing false.
maxredirects (default: 5): sets the
number of allowed redirects to minimize looping due to
misconfigured servers. When the number is exceeded, the result
of the last redirect is returned.
keepalive (default: false): configures
each request to keep the connection open (unless closed by
the server) and reuse for the next request to the same
host. This option is disabled when SSL connection is used.
The mapping of hosts and their sockets is stored in a
table assigned to the `keepalive` field itself, so it can
be passed to the next call. If the table includes the
`close` field set to a true value, then the connection is
closed after the request is made and the host is removed
from the mapping table.
Mon, 29 Mar 2021 15:37:13 GMT.
See formathttpdatetime.c.
ParseIp for the inverse
operation.
Here a small usage sample:
>: secret1 = GetRandomBytes(32)
>: public1 = Curve25519(secret1, "\9")
>: secret2 = GetRandomBytes(32)
>: public2 = Curve25519(secret2, "\9")
>: shared_key1 = Curve25519(secret1, public2)
>: shared_key2 = Curve25519(secret2, public1)
>: shared_key1 == shared_key2
true
0x01020304,31337 would
represent 1.2.3.4:31337. This is the same
as GetClientAddr except
it will use the ip:port from the X-Forwarded-For header
when the IP returned by GetClientAddr is a trusted IP (returns true
when checked against IsTrustedIp). When multiple addresses are
present in the header, the last/right-most address is used.
Returns nil if the address doesn't parse as IPv4 value
(e.g. because it is misformatted or an IPv6 address).
0x01020304,31337 would
represent 1.2.3.4:31337. Please consider
using GetRemoteAddr
instead, since the latter takes into consideration reverse proxy
scenarios.
0x01020304,8080 would
represent 1.2.3.4:8080. If -p 0 was
supplied as the listening port, then the port in this string will be
whatever number the operating system assigned.
Date header, which is now, give or take a second.
The returned value is a UNIX timestamp.
name is case-insensitive. The
header value is returned as a canonical UTF-8 string,
with leading and trailing whitespace trimmed, which was decoded from
ISO-8859-1, which is guaranteed to not have C0/C1 control sequences,
with the exception of the tab character. Leading and trailing
whitespace is automatically removed. In the event that the client
suplies raw UTF-8 in the HTTP message headers, the original UTF-8
sequence can be losslessly restored by counter-intuitively recoding
the returned string back to Latin1. If the requested header is
defined by the RFCs as storing comma-separated values (e.g. Allow,
Accept-Encoding) and the field name occurs multiple times in the
message, then this function will fold those multiple entries into a
single string.
GetHeader API if
possible since it does a better job abstracting these issues.
kLogDebug > kLogVerbose > kLogInfo > kLogWarn > kLogError > kLogFatal.
This can return:
"LINUX"
"METAL"
"WINDOWS"
"XNU"
"NETBSD"
"FREEBSD"
"OPENBSD"
This can return:
"X86_64" for Intel and AMD systems
"AARCH64" for ARM64, M1, and Raspberry Pi systems
"POWERPC64" for OpenPOWER Raptor Computing Systems
GET, HEAD, or POST in which
case redbean normalizes this value to its uppercase form. Anything else
that the RFC classifies as a "token" string is accepted too, which might
contain characters like &".
application/x-www-form-urlencoded message body in the
order they were received. This may contain duplicates. The inner array
will have either one or two items, depending on whether or not the
equals sign was used.
"/". It is further guaranteed that
no "//" or "/." exists in the path. The
returned value is returned as a UTF-8 string which was decoded from
ISO-8859-1. We assume that percent-encoded characters were supplied by
the client as UTF-8 sequences, which are returned exactly as the client
supplied them, and may therefore may contain overlong sequences, control
codes, NUL characters, and even numbers which have been
banned by the IETF. redbean takes those things into consideration when
performing path safety checks. It is the responsibility of the caller to
impose further restrictions on validity, if they're desired.
SetStatus call) or
nil if the status hasn't been set yet.
Host or X-Forwarded-Host headers, if they exist, and possibly a scheme too if redbean is being used as an HTTP proxy server. In the future this API might change to return an object instead.
9 for HTTP/0.9, 10
for HTTP/1.0, or 11
for HTTP/1.1.application/x-www-form-urlencoded message body.
/ listing page to not display any paths beginning with prefix. This function should only be called from /.init.lua.
IsCompressed
-D flag was used. If slurping large file into memory is a concern, then consider using ServeAsset which can serve directly off disk.
GetLogLevel. If redbean is
running in interactive mode, then this will log to the console. If
redbean is running as a daemon or the -L LOGFILE flag is
passed, then this will log to the file. Reasonable values
for level
are kLogDebug > kLogVerbose > kLogInfo > kLogWarn > kLogError > kLogFatal.
The logger emits timestamps in the local timezone with microsecond
precision. If log entries are emitted more frequently than once per
second, then the log entry will display a delta timestamp, showing how
much time has elapsed since the previous log entry. This behavior is
useful for quickly measuring how long various portions of your code take
to execute.
Mon, 29 Mar 2021 15:37:13 GMT to a UNIX timestamp. See parsehttpdatetime.c.
An object containing the following fields is returned:
- scheme is a string, e.g. "http"
- user is the username string, or nil if absent
- pass is the password string, or nil if absent
- host is the hostname string, or nil if url was a path
- port is the port string, or nil if absent
- path is the path string, or nil if absent
- params is the URL paramaters, e.g. /?a=b&c would be
represented as the data structure {{"a", "b"}, {"c"}, ...}
- fragment is the stuff after the # character
flags may have:
- kUrlPlus to turn + into space
- kUrlLatin1 to transcode ISO-8859-1 input into UTF-8
This parser is charset agnostic. Percent encoded bytes are decoded for all fields. Returned values might contain things like NUL characters, spaces, control codes, and non-canonical encodings. Absent can be discerned from empty by checking if the pointer is set.
There's no failure condition for this routine. This is a
permissive parser. This doesn't normalize path segments like
. or .. so use IsAcceptablePath() to
check for those. No restrictions are imposed beyond that which
is strictly necessary for parsing. All the data that is
provided will be consumed to the one of the fields. Strict
conformance is enforced on some fields more than others, like
scheme, since it's the most non-deterministically defined
field of them all.
Please note this is a URL parser, not a URI parser. Which means we support everything the URI spec says we should do except for the things we won't do, like tokenizing path segments into an array and then nesting another array beneath each of those for storing semicolon parameters. So this parser won't make SIP easy. What it can do is parse HTTP URLs and most URIs like data:opaque, better in fact than most things which claim to be URI parsers.
".", ".." or "//" segments
See isacceptablepath.c
"." or ".." segments
See isreasonablepath.c
ParseUrl. The output will always
be correctly formatted. The exception is if illegal characters are
supplied in the scheme field, since there's no way of escaping those.
Opaque parts are escaped as though they were paths, since many URI
parsers won't understand things like an unescaped question mark in path.
"1.2.3.4" → 0x01020304,
or returns -1 for invalid inputs. See also FormatIp
for the inverse operation.
GetComment (deprecated).
GetLastModifiedTime (deprecated).
-D flag is used)
-D flag is used)
-d flag was passed to redbean.
-G flag if called from .init.lua for setgid()
-D flag if called from .init.lua for
overlaying local file system directories. This may be called
multiple times. The first directory programmed is preferred. These
currently do not show up in the index page listing.
This call also modifies package.path value to
either prepend the added directory in front of the default
path (if found) or to append it (in all other cases). If no
directory is provided, then a table with previously set
directories is returned.
-m flag if called from .init.lua for logging message
headers only.
-b flag if called from .init.lua for
logging message bodies as part of POST / PUT / etc. requests.
-L flag if called from .init.lua for
setting the log file path on the local file system. It's created if
it doesn't exist. This is called before de-escalating the user /
group id. The file is opened in append only mode. If the disk runs
out of space then redbean will truncate the log file if has access
to change the log file after daemonizing.
-P flag if called from .init.lua for
setting the pid file path on the local file system. It's useful for
reloading daemonized redbean using kill -HUP $(cat /var/run/redbean.pid)
or terminating redbean with kill $(cat /var/run/redbean.pid) which
will gracefully terminate all clients. Sending the TERM signal twice will cause a
forceful shutdown, which might make someone with a slow internet connection who's
downloading big files unhappy.
This function reads file data from local file system. Zip file assets can be accessed using the `/zip/...` prefix.
i and j may be used to slice a
substring in filename. These parameters are
1-indexed and behave consistently with Lua's string.sub() API.
For example:
assert(Barf('x.txt', 'abc123')) assert(assert(Slurp('x.txt', 2, 3)) == 'bc')
This function is uninterruptible so unix.EINTR
errors will be ignored. This should only be a concern if
you've installed signal handlers. Use the UNIX API if you need
to react to it.
This function writes to the local file system.
mode defaults to 0644. This parameter is ignored when
flags doesn't have unix.O_CREAT.
flags defaults to unix.O_TRUNC | unix.O_CREAT.
offset is 1-indexed and may be used to overwrite arbitrary
slices within a file when used in conjunction with flags=0.
For example:
assert(Barf('x.txt', 'abc123')) assert(Barf('x.txt', 'XX', 0, 0, 3)) assert(assert(Slurp('x.txt', 1, 6)) == 'abXX23')
milliseconds is not
specified, then the current interval is returned.
/.init.lua.
Server header, as well as the <h1> title on the / listing page. The brand string needs to be a UTF-8 value that's encodable as ISO-8859-1. If the brand is changed to something other than redbean, then the promotional links will be removed from the listing page too. This function should only be called from /.init.lua.
Cache-Control and Expires header generation for static asset serving. A negative value will disable the headers. Zero means don't cache. Greater than zero asks public proxies and
browsers to cache for a given number of seconds. The directive
value is added to the Cache-Control header when specified (with
"must-revalidate" provided by default) and can be set to an empty
string to remove the default value.
This function should only be called from /.init.lua.
GetServerAddr or the -z flag to stdout.
301, 302, 307,
or 308 then a redirect response will be sent to the client.
This function should only be called
from /.init.lua.
-C flag if called
from .init.lua,
e.g. ProgramCertificate(LoadAsset("/.sign.crt")) for zip
loading or
ProgramCertificate(Slurp("/etc/letsencrypt.lol/fullchain.pem"))
for local file system only. This function is not available in unsecure
mode.
-K flag if called
from .init.lua, e.g.
ProgramPrivateKey(LoadAsset("/.sign.key")) for zip loading or
ProgramPrivateKey(Slurp("/etc/letsencrypt/privkey.pem"))
for local file system only. This function is not available in unsecure
mode.
key may contain 1..32 bytes of random
binary data and identity is usually a short plaintext string. The
first time this function is called, the preshared key will be added
to both the client and the server SSL configs. If it's called
multiple times, then the remaining keys will be added to the server,
which is useful if you want to assign separate keys to each client,
each of which needs a separate identity too. If this function is
called multiple times with the same identity string, then the latter
call will overwrite the prior. If a preshared key is supplied and no
certificates or key-signing-keys are programmed, then redbean won't
bother auto-generating any serving certificates and will instead use
only PSK ciphersuites. This function is not available in unsecure
mode.
-j flag. Tuning this option alone does not preclude
the possibility of unsecured HTTP clients, which can be disabled
using ProgramSslRequired(). This
function can only be called from .init.lua. This
function is not available in unsecure mode.
-J flag. Fetch() is still
allowed to make outbound HTTP requests. This function can only be
called from .init.lua. This function is not available
in unsecure mode.
ECDHE-ECDSA-AES256-GCM-SHA384 ECDHE-ECDSA-AES128-GCM-SHA256 ECDHE-ECDSA-CHACHA20-POLY1305-SHA256 ECDHE-PSK-AES256-GCM-SHA384 ECDHE-PSK-AES128-GCM-SHA256 ECDHE-PSK-CHACHA20-POLY1305-SHA256 ECDHE-RSA-AES256-GCM-SHA384 ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-CHACHA20-POLY1305-SHA256 DHE-RSA-AES256-GCM-SHA384 DHE-RSA-AES128-GCM-SHA256 DHE-RSA-CHACHA20-POLY1305-SHA256 ECDHE-ECDSA-AES128-CBC-SHA256 ECDHE-RSA-AES256-CBC-SHA384 ECDHE-RSA-AES128-CBC-SHA256 DHE-RSA-AES256-CBC-SHA256 DHE-RSA-AES128-CBC-SHA256 ECDHE-PSK-AES256-CBC-SHA384 ECDHE-PSK-AES128-CBC-SHA256 ECDHE-ECDSA-AES256-CBC-SHA ECDHE-ECDSA-AES128-CBC-SHA ECDHE-RSA-AES256-CBC-SHA ECDHE-RSA-AES128-CBC-SHA DHE-RSA-AES256-CBC-SHA DHE-RSA-AES128-CBC-SHA ECDHE-PSK-AES256-CBC-SHA ECDHE-PSK-AES128-CBC-SHA RSA-AES256-GCM-SHA384 RSA-AES128-GCM-SHA256 RSA-AES256-CBC-SHA256 RSA-AES128-CBC-SHA256 RSA-AES256-CBC-SHA RSA-AES128-CBC-SHA PSK-AES256-GCM-SHA384 PSK-AES128-GCM-SHA256 PSK-CHACHA20-POLY1305-SHA256 PSK-AES256-CBC-SHA384 PSK-AES128-CBC-SHA256 PSK-AES256-CBC-SHA PSK-AES128-CBC-SHA ECDHE-RSA-3DES-EDE-CBC-SHA DHE-RSA-3DES-EDE-CBC-SHA ECDHE-PSK-3DES-EDE-CBC-SHA RSA-3DES-EDE-CBC-SHA PSK-3DES-EDE-CBC-SHA
When redbean is run on an old (or low-power) CPU that doesn't have the AES-NI instruction set (Westmere c. 2010) then the default ciphersuite is tuned automatically to favor the ChaCha20 Poly1305 suites.
The names above are canonical to redbean. They were programmatically simplified from the official IANA names. This function will accept the IANA names too. In most cases it will accept the OpenSSL and GnuTLS naming convention as well.
This function is not available in unsecure mode.
OnHttpRequest
handler, since that overrides the serving path entirely. So if the
handler decides it doesn't want to do anything, it can simply call
this function, to hand over control back to the redbean core. By
default, the host and path arguments are supplied from the resolved
GetUrl value. This handler always resolves, since it will generate a
404 Not Found response if redbean couldn't find an appropriate
endpoint.
Route, except it only implements
the subset of request routing needed for serving virtual-hosted
assets, where redbean tries to prefix the path with the hostname
when looking up a file. This function returns true if the request
was resolved. If it was resolved, then your OnHttpRequest request
handler can still set additional headers.
Route, except it only implements
the subset of request routing needed for serving assets. This
function returns true if the request was resolved. If it was
resolved, then your OnHttpRequest
request handler can still set additional headers.
-D is used. This function is mutually
exclusive with SetStatus
and ServeError.
SetStatus
and ServeAsset.
level
are kLogDebug > kLogVerbose > kLogInfo > kLogWarn > kLogError > kLogFatal.
>: Decimate('\xff\xff\x00\x00\xff\xff\x00\x00\xff\xff\x00\x00')
"\xff\x00\xff\x00\xff\x00"
This is very fast if SSSE3 is available (Intel 2004+ / AMD 2011+).
>: Deflate("hello") "\xcbH\xcd\xc9\xc9\x07\x00" >: Inflate("\xcbH\xcd\xc9\xc9\x07\x00", 5) "hello"
The output format is raw DEFLATE that's suitable for embedding into formats like a ZIP file. It's recommended that, like ZIP, you also store separately a Crc32() checksum in addition to the original uncompressed size.
level is the compression level, which defaults to 7. The
max is 9. Lower numbers go faster (4 for instance is a sweet spot) and
higher numbers go slower but have better compression.
This function performs the inverse of Deflate(). It's recommended that you perform a Crc32() check on the output string after this function succeeds.
maxoutsize is the uncompressed size, which should be
known. However, it is permissable (although not advised) to specify
some large number in which case (on success) the byte length of the
output string may be less than maxoutsize.
The first value returned is the average number of nanoseconds that
func needed to execute. Nanoseconds are computed from
RDTSC tick counts, using an approximation that's measured beforehand
with the unix.clock_gettime()
function.
The ticks result is the canonical average number of clock ticks.
This subroutine will subtract whatever the overhead happens to be for benchmarking a function that does nothing. This overhead value will be reported in the result.
tries indicates if your microbenchmark needed to be repeated,
possibly because your system is under load and the benchmark was
preempted by the operating system, or moved to a different core.
"0". Otherwise the resulting
value will be the zero-prefixed octal string. The result is currently
modulo 2^64. Negative numbers are converted to unsigned.
"0". Otherwise the
resulting value will be the "0x"-prefixed hex string. The
result is currently modulo 2^64. Negative numbers are converted to
unsigned.
"0". Otherwise the resulting
value will be the "0b"-prefixed binary str. The result is
currently modulo 2^64. Negative numbers are converted to unsigned.
This function first checks if hostname is already an IP address, in
which case it returns the result
of ParseIp. Otherwise, it checks
HOSTS.TXT on the local system and returns the first IPv4 address
associated with hostname. If no such entry is found, a DNS lookup is
performed using the system configured (e.g. /etc/resolv.conf) DNS
resolution service. If the service returns multiple IN A records
then only the first one is returned.
The returned address is word-encoded in host endian order. For
example, 1.2.3.4 is encoded as 0x01020304.
The FormatIp function may be
used to turn this value back into a string.
If no IP address could be found, then nil is returned alongside a
string of unspecified format describing the error. Calls to this
function may be wrapped in assert() if an exception is
desired.
If the ProgramTrustedIp() function has NOT been called then redbean will consider the networks 127.0.0.0/8, 10.0.0.0/8, 172.16.0.0/12, and 192.168.0.0/16 to be trustworthy too. If ProgramTrustedIp() HAS been called at some point earlier in your redbean's lifecycle, then it'll trust the IPs and network subnets you specify instead.
The network interface addresses used by the host machine are always considered trustworthy, e.g. 127.0.0.1. This may change soon, if we decide to export a GetHostIps() API which queries your NIC devices.
This function may be used to configure the IsTrustedIp() function which is how redbean determines if a client is allowed to send us headers like X-Forwarded-For (cf GetRemoteAddr vs. GetClientAddr) without them being ignored. Trusted IPs is also how redbean turns off token bucket rate limiting selectively, so be careful. Here's an example of how you could trust all of Cloudflare's IPs:
ProgramTrustedIp(ParseIp("103.21.244.0"), 22);
ProgramTrustedIp(ParseIp("103.22.200.0"), 22);
ProgramTrustedIp(ParseIp("103.31.4.0"), 22);
ProgramTrustedIp(ParseIp("104.16.0.0"), 13);
ProgramTrustedIp(ParseIp("104.24.0.0"), 14);
ProgramTrustedIp(ParseIp("108.162.192.0"), 18);
ProgramTrustedIp(ParseIp("131.0.72.0"), 22);
ProgramTrustedIp(ParseIp("141.101.64.0"), 18);
ProgramTrustedIp(ParseIp("162.158.0.0"), 15);
ProgramTrustedIp(ParseIp("172.64.0.0"), 13);
ProgramTrustedIp(ParseIp("173.245.48.0"), 20);
ProgramTrustedIp(ParseIp("188.114.96.0"), 20);
ProgramTrustedIp(ParseIp("190.93.240.0"), 20);
ProgramTrustedIp(ParseIp("197.234.240.0"), 22);
ProgramTrustedIp(ParseIp("198.41.128.0"), 17);
Although you might want consider trusting redbean's open source freedom embracing solution to DDOS protection instead!
Imagine you have 232 buckets, one for each IP address. Each bucket can hold about 127 tokens. Every second a background worker puts one token in each bucket. When a TCP client socket is opened, it takes a token from its bucket, and then proceeds. If the bucket holds only a third of its original tokens, then redbean sends them a 429 warning. If the client ignores this warning and keeps sending requests, until there's no tokens left, then the banhammer finally comes down.
function OnServerStart() ProgramTokenBucket() ProgramTrustedIp(ParseIp('x.x.x.x'), 32) assert(unix.setrlimit(unix.RLIMIT_NPROC, 1000, 1000)) end
This model of network rate limiting generously lets people "burst" a
tiny bit. For example someone might get a strong craving for content
and smash the reload button in Chrome 64 times in a few seconds. But
since the client only get 1 new token per second, they'd better cool
their heels for a few minutes after doing that. This amount of burst
can be altered by choosing the reject / ignore / ban threshold
arguments. For example, if the reject parameter is set to 126 then
no bursting is allowed, which probably isn't a good idea.
redbean is programmed to acquire a token immediately after accept()
is called from the main server process, which is well before fork()
or read() or any Lua code happens. redbean then takes action, based
on the token count, which can be accept / reject / ignore / ban. If
redbean determines a ban is warrented, then 4-byte datagram is sent
to the unix domain socket /var/run/blackhole.sock which should be
operated using the blackholed program we distribute separately.
The trick redbean uses on Linux for example is insert rules in your raw prerouting table. redbean is very fast at the application layer so the biggest issue we've encountered in production is are kernels themselves, and programming the raw prerouting table dynamically is how we solved that.
replenish is the number of times per second a token should be
added to each bucket. The default value is 1 which means one token
is granted per second to all buckets. The minimum value is 1/3600
which means once per hour. The maximum value for this setting is
1e6, which means once every microsecond.
cidr is the specificity of judgement. Since creating 2^32 buckets
would need 4GB of RAM, redbean defaults this value to 24 which means
filtering applies to class c network blocks (i.e. x.x.x.*), and your
token buckets only take up 2^24 bytes of RAM (16MB). This can be set
to any number on the inclusive interval [8,32], where having a lower
number means you use less ram/cpu, but splash damage applies more to
your clients; whereas higher numbers means more ram/cpu usage, while
ensuring rate limiting only applies to specific compromised actors.
reject is the token count or treshold at which redbean should send
429 Too Many Request warnings to the client. Permitted values can be
anywhere between -1 and 126 inclusively. The default value is 30 and
-1 means to disable (assuming AcquireToken() will be used).
ignore is the token count or treshold, at which redbean should try
simply ignoring clients and close the connection without logging any
kind of warning, and without sending any response. The default value
for this setting is MIN(reject / 2, 15). This must be less than or
equal to the reject setting. Allowed values are [-1,126] where you
can use -1 as a means of disabling ignore.
ban is the token count at which redbean should report IP addresses
to the blackhole daemon via a unix-domain socket datagram so they'll
get banned in the kernel routing tables. redbean's default value for
this setting is MIN(ignore / 10, 1). Permitted values are [-1,126]
where -1 may be used as a means of disabling the ban feature.
This function throws an exception if the constraints described above are not the case. Warnings are logged should redbean fail to connect to the blackhole daemon, assuming it hasn't been disabled. It's safe to use load balancing tools when banning is enabled, since you can't accidentally ban your own network interface addresses, loopback ips, or ProgramTrustedIp() addresses where these rate limits don't apply.
It's assumed will be called from the .init.lua global scope although it could be used in interpreter mode, or from a forked child process in which case the only processes that'll have ability to use it will be that same process, and any descendent processes. This function is only able to be called once.
This feature is not available in unsecure mode.
This routine atomically acquires a single token for
an ip address. The return value is the token count
before the subtraction happened. No action is taken based on the
count, since the caller will decide.
ip should be an IPv4 address and this defaults to
GetClientAddr(), although other interpretations of its meaning are
possible.
Your token buckets are stored in shared memory so this can be called from multiple forked processes. which operate on the same values.
ProgramTokenBucket() needs to be called beforehand. The default settings will blackhole automatically, during the accept() loop based on the banned threshold. However if your Lua code calls AcquireToken() manually, then you'll need this function to take action on the returned values.
This function returns true if a datagram could be sent sucessfully. Otherwise false is returned, which can happen if blackholed isn't running, or if a lot of processes are sending messages to it and the operation would have blocked.
It's assumed that the blackholed service is running locally in the background.
Please refer to the LuaSQLite3 Documentation.
For example, you could put the following in your /.init.lua file:
sqlite3 = require 'lsqlite3' db = sqlite3.open_memory() db:exec[[ CREATE TABLE test ( id INTEGER PRIMARY KEY, content TEXT ); INSERT INTO test (content) VALUES ('Hello World'); INSERT INTO test (content) VALUES ('Hello Lua'); INSERT INTO test (content) VALUES ('Hello Sqlite3'); ]]
Then, your Lua server pages or OnHttpRequest handler may perform SQL queries by accessing the db global. The performance is good too, at about 400k qps.
for row in db:nrows("SELECT * FROM test") do Write(row.id.." "..row.content.."<br>") end
Warning: Please read
How To Corrupt An
SQLite Database File because those same rules apply to redbean.
One recommendation worth paying attention to is that SQLite
connections can't cross fork() boundaries, and redbean is a forking
web server. The simplest solution to solving this, is to create your
database handles inside your OnWorkerStart() hook.
The above example is only appropriate for read-only databases. When setting up a mutable SQLite database, it's important to consider that redbean is a forking web server, and SQLite handles shouldn't cross fork boundaries. That means you need to create your SQLite database object lazily from within the client process, and SQLite makes this very fast. One approach that's worked well for us on Linux, is to do that using the write-ahead log. For example:
re = require 're' sqlite3 = require 'lsqlite3' reNumberPath = re.compile[[^/([0-9][0-9]*)$]] function SetupSql() if not db then db = sqlite3.open('redbean.sqlite3') db:busy_timeout(1000) db:exec[[PRAGMA journal_mode=WAL]] db:exec[[PRAGMA synchronous=NORMAL]] getBarStmt = db:prepare[[ SELECT foo FROM Bar WHERE id = ? ]] end end local function GetBar(id) if not getBarStmt then Log(kLogWarn, 'prepare failed: ' .. db:errmsg()) return nil end getBarStmt:reset() getBarStmt:bind(1, id) for bar in getBarStmt:nrows() do return bar end return nil end function OnHttpRequest() SetupSql() _, id = reNumberPath:search(GetPath()) if id then bar = GetBar(id) SetHeader('Content-Type', 'text/plain; charset=utf-8') Write(string(bar.foo)) return end Route() SetHeader('Content-Language', 'en-US') end
redbean supports a subset of what's defined in the upstream LuaSQLite3 project. Most of the unsupported APIs relate to pointers and database notification hooks.
This module exposes an API for POSIX regular expressions which enable you to validate input, search for substrings, extract pieces of strings, etc. Here's a usage example:
# Example IPv4 Address Regular Expression (see also ParseIP)
p = assert(re.compile([[^([0-9]{1,3})\.([0-9]{1,3})\.([0-9]{1,3})\.([0-9]{1,3})$]]))
m,a,b,c,d = assert(p:search(𝑠))
if m then
print("ok", tonumber(a), tonumber(b), tonumber(c), tonumber(d))
else
print("not ok")
end
This is a shorthand notation roughly equivalent to:
preg = re.compile(regex) patt = preg:search(re, text)
flags defaults to zero and may have any of:
re.BASIC
re.ICASE
re.NEWLINE
re.NOSUB
re.NOTBOL
re.NOTEOL
This has exponential complexity. Please use re.compile() to compile
your regular expressions once from /.init.lua. This API
exists for convenience. This isn't recommended for prod.
This uses POSIX extended syntax by default.
flags defaults to zero and may have any of:
re.BASIC
re.ICASE
re.NEWLINE
re.NOSUB
This has an O(2^𝑛) cost. Consider compiling regular
expressions once from your /.init.lua file.
If regex is an untrusted user value, then
unix.setrlimit should be used to impose cpu and memory
quotas for security.
This uses POSIX extended syntax by default.
Returns nothing (nil) if the pattern doesn't match anything.
Otherwise it pushes the matched substring and any
parenthesis-captured values too. Flags may contain re.NOTBOL
or re.NOTEOL to indicate whether or not text should
be considered at the start and/or end of a line.
`flags` defaults to zero and may have any of:
re.NOTBOL
re.NOTEOL
This has an O(𝑛) cost.
re.Errno:doc()
([0-9]*)\.([0-9]*)\.([0-9]*)\.([0-9]*) whereas
with basic syntax it would look
like \([0-9]*\)\.\([0-9]*\)\.\([0-9]*\)\.\([0-9]*\).
This flag may only be used
with re.compile
and re.search.
[a-z] will mean the same thing
as [A-Za-z]. This flag may only be used
with re.compile
and re.search.
re.search to only report
success and failure. This is reported via the API by returning empty
string for success. This flag may only be used
with re.compile
and re.search.
re.search
and regex_t*:search.
re.search
and regex_t*:search.
]
)
}
{}
The path module may be used to manipulate unix paths.
Note that we use unix paths on Windows. For example, if you have a
path like C:\foo\bar then it should be
/c/foo/bar with redbean. It should also be noted the
unix module is more permissive when using Windows paths, where
translation to win32 is very light.
path │ dirname ─────────────────── . │ . .. │ . / │ / usr │ . /usr/ │ / /usr/lib │ /usr /usr/lib/ │ /usr
path │ basename ───────────────────── . │ . .. │ .. / │ / usr │ usr /usr/ │ usr /usr/lib │ lib /usr/lib/ │ lib
x │ y │ joined ───────────────────────────────── / │ / │ / /usr │ lib │ /usr/lib /usr/ │ lib │ /usr/lib /usr/lib │ /lib │ /lib
You may specify 1+ arguments.
Specifying no arguments will raise an error. If nil arguments are specified, then they're skipped over. If exclusively nil arguments are passed, then nil is returned. Empty strings behave similarly to nil, but unlike nil may coerce a trailing slash.
This function is inclusive of regular files, directories, and special files. Symbolic links are followed are resolved. On error, false is returned.
Symbolic links are not followed. On error, false is returned.
Symbolic links are not followed. On error, false is returned.
Symbolic links are not followed. On error, false is returned.
This module may be used to get city/country/asn/etc from IPs, e.g.
-- .init.lua maxmind = require 'maxmind' asndb = maxmind.open('/usr/local/share/maxmind/GeoLite2-ASN.mmdb') -- request handler as = asndb:lookup(GetRemoteAddr()) if as then asnum = as:get('autonomous_system_number') asorg = as:get('autonomous_system_organization') Write(EscapeHtml(asnum)) Write(' ') Write(EscapeHtml(asorg)) end
The database file is distributed by MaxMind. You need to sign up on their website to get a free copy. The database has a generalized structure. For a concrete example of how this module may be used, please see maxmind.lua in redbean-demo.com.
This is an experimental module that, like the maxmind module, gives you insight into what kind of device is connecting to your redbean. This module can help you protect your redbean because it provides tools for identifying clients that misrepresent themselves. For example the User-Agent header might report itself as a Windows computer when the SYN packet says it's a Linux computer.
function OnServerListen(fd, ip, port) unix.setsockopt(fd, unix.SOL_TCP, unix.TCP_SAVE_SYN, true) return false end function OnClientConnection(ip, port, serverip, serverport) fd = GetClientFd() syn = unix.getsockopt(fd, unix.SOL_TCP, unix.TCP_SAVED_SYN) end function OnHttpRequest() Log(kLogInfo, "client is running %s and reports %s" % { finger.GetSynFingerOs(finger.FingerSyn(syn)), GetHeader('User-Agent')}) Route() end
The following functions are provided.
This returns a hash-like magic number that reflects the SYN packet structure, e.g. ordering of options, maximum segment size, etc. We make no guarantees this hashing algorithm won't change as we learn more about the optimal way to fingerprint, so be sure to save your syn packets too if you're using this feature, in case they need to be rehashed in the future.
This function is nil/error propagating.
If synfinger is a known hard-coded magic number, then
one of the following strings may be returned:
"LINUX"
"WINDOWS"
"XNU"
"NETBSD"
"FREEBSD"
"OPENBSD"
If this function returns nil, then one thing you can do to help is file an issue and share with us your SYN packet specimens. The way we prefer to receive them is in EncodeLua(syn_packet_bytes) format along with details on the operating system which you must know.
The layout looks as follows:
TTL:OPTIONS:WSIZE:MSS
The TTL, WSIZE, and MSS fields are unsigned decimal fields.
The OPTIONS field communicates the ordering of the commonly used
subset of tcp options. The following character mappings are defined.
TCP options not on this list will be ignored.
E: End of Option list
N: No-Operation
M: Maxmimum Segment Size
K: Window Scale
O: SACK Permitted
A: SACK
e: Echo (obsolete)
r: Echo reply (obsolete)
T: Timestamps
This function is nil/error propagating.
This module implements a password hashing algorithm based on blake2b that won the Password Hashing Competition.
It can be used to securely store user passwords in your SQLite database, in a way that destroys the password, but can be verified by regenerating the hash again the next time the user logs in. Destroying the password is important, since if your database is compromised, the bad guys won't be able to use rainbow tables to recover the plain text of the passwords.
Argon2 achieves this security by being expensive to compute. Care should be taken in choosing parameters, since an HTTP endpoint that uses Argon2 can just as easily become a denial of service vector. For example, you may want to consider throttling your login endpoint.
This is consistent with the README of the reference implementation:
>: assert(argon2.hash_encoded("password", "somesalt", {
variant = argon2.variants.argon2_i,
m_cost = 65536,
hash_len = 24,
parallelism = 4,
t_cost = 2,
}))
"$argon2i$v=19$m=65536,t=2,p=4$c29tZXNhbHQ$RdescudvJCsgt3ub+b+dWRWJTmaaJObG"
pass is the secret value to be encoded.
salt is a nonce value used to hash the string.
config.m_cost is the memory hardness in kibibytes, which defaults
to 4096 (4 mibibytes). It's recommended that this be tuned upwards.
config.t_cost is the number of iterations, which defaults to 3.
config.parallelism is the parallelism factor, which defaults to 1.
config.hash_len is the number of desired bytes in hash output,
which defaults to 32.
config.variant may be:
argon2.variants.argon2_id blend of other two methods [default]
argon2.variants.argon2_i maximize resistance to side-channel attacks
argon2.variants.argon2_d maximize resistance to gpu cracking attacks
>: argon2.verify("$argon2i$v=19$m=65536,t=2,p=4$c29tZXNhbHQ$RdescudvJCsgt3ub+b+dWRWJTmaaJObG", "password")
true
This module exposes the low-level System Five system call interface. This module works on all supported platforms, including Windows NT.
Returns a file descriptor integer that needs to be closed, e.g.
fd = assert(unix.open('/etc/passwd', unix.O_RDONLY)) print(unix.read(fd)) unix.close(fd)
flags should have one of:
O_RDONLY: open for reading (default)
O_WRONLY: open for writing
O_RDWR: open for reading and writing
The following values may also be OR'd into flags:
O_CREAT create file if it doesn't exist
O_TRUNC automatic ftruncate(fd,0) if exists
O_CLOEXEC automatic close() upon execve()
O_EXCL exclusive access (see below)
O_APPEND open file for append only
O_NONBLOCK asks read/write to fail with EAGAIN rather than block
O_DIRECT it's complicated (not supported on Apple and OpenBSD)
O_DIRECTORY useful for stat'ing (hint on UNIX but required on NT)
O_NOFOLLOW fail if it's a symlink (zero on Windows)
O_DSYNC it's complicated (zero on non-Linux/Apple)
O_RSYNC it's complicated (zero on non-Linux/Apple)
O_PATH it's complicated (zero on non-Linux)
O_VERIFY it's complicated (zero on non-FreeBSD)
O_SHLOCK it's complicated (zero on non-BSD)
O_EXLOCK it's complicated (zero on non-BSD)
O_NOATIME don't record access time (zero on non-Linux)
O_RANDOM hint random access intent (zero on non-Windows)
O_SEQUENTIAL hint sequential access intent (zero on non-Windows)
O_COMPRESSED ask fs to abstract compression (zero on non-Windows)
O_INDEXED turns on that slow performance (zero on non-Windows)
There are three regular combinations for the above flags:
O_RDONLY: Opens existing file for reading. If it
doesn't exist then nil is returned and errno will
be ENOENT (or in some
other cases ENOTDIR).
O_WRONLY|O_CREAT|O_TRUNC: Creates file. If it already
exists, then the existing copy is destroyed and the opened
file will start off with a length of zero. This is the
behavior of the traditional creat() system call.
O_WRONLY|O_CREAT|O_EXCL: Create file only if doesn't exist
already. If it does exist then nil is returned along with
errno set to EEXIST.
dirfd defaults to to unix.AT_FDCWD and may optionally be set to
a directory file descriptor to which path is relative.
Returns ENOENT
if path doesn't exist.
Returns ENOTDIR
if path contained a directory component that wasn't a
directory.
This function should never be called twice for the same file descriptor, regardless of whether or not an error happened. The file descriptor is always gone after close is called. So it technically always succeeds, but that doesn't mean an error should be ignored. For example, on NFS a close failure could indicate data loss.
Closing does not mean that scheduled i/o operations have been completed. You'd need to use fsync() or fdatasync() beforehand to ensure that. You shouldn't need to do that normally, because our close implementation guarantees a consistent view, since on systems where it isn't guaranteed (like Windows) close will implicitly sync.
File descriptors are automatically closed on exit().
Returns EBADF if fd wasn't valid.
Returns EINTR possibly maybe.
Returns EIO if an i/o error occurred.
This function returns empty string on end of file. The exception is
if bufsiz is zero, in which case an empty returned
string means the file descriptor works.
_Exit(exitcode) on the process. This will
immediately halt the current process. Memory will be freed. File
descriptors will be closed. Any open connections it owns will be
reset. This function never returns.
This allocates and constructs the C/C++ environ
variable as a Lua table consisting of string keys and string values.
This data structure preserves casing. On Windows NT, by convention, environment variable keys are treated in a case-insensitive way. It is the responsibility of the caller to consider this.
This data structure preserves valueless variables. It's possible on both UNIX and Windows to have an environment variable without an equals, even though it's unusual.
This data structure preserves duplicates. For example, on Windows, there's some irregular uses of environment variables such as how the command prompt inserts multiple environment variables with empty string as keys, for its internal bookkeeping.
This system call returns twice. The parent process gets the nonzero pid. The child gets zero.
Here's a simple usage example of creating subprocesses, where we fork off a child worker from a main process hook callback to do some independent chores, such as sending an HTTP request back to redbean.
-- as soon as server starts, make a fetch to the server -- then signal redbean to shutdown when fetch is complete local onServerStart = function() if assert(unix.fork()) == 0 then local ok, headers, body = Fetch('http://127.0.0.1:8080/test') unix.kill(unix.getppid(), unix.SIGTERM) unix.exit(0) end end OnServerStart = onServerStart
We didn't need to use wait() here, because (a) we want redbean to go back to what it was doing before as the Fetch() completes, and (b) redbean's main process already has a zombie collector. However it's a moot point, since once the fetch is done, the child process then asks redbean to gracefully shutdown by sending SIGTERM its parent.
This is actually a situation where we *must* use fork, because the purpose of the main redbean process is to call accept() and create workers. So if we programmed redbean to use the main process to send a blocking request to itself instead, then redbean would deadlock and never be able to accept() the client.
While deadlocking is an extreme example, the truth is that latency issues can crop up for the same reason that just cause jitter instead, and as such, can easily go unnoticed. For example, if you do something that takes longer than a few milliseconds from inside your redbean heartbeat, then that's a few milliseconds in which redbean is no longer concurrent, and tail latency is being added to its ability to accept new connections. fork() does a great job at solving this.
If you're not sure how long something will take, then when in doubt, fork off a process. You can then report its completion to something like SQLite. Redbean makes having lots of processes cheap. On Linux they're about as lightweight as what heavyweight environments call greenlets. You can easily have 10,000 Redbean workers on one PC.
Here's some benchmarks for fork() performance across platforms:
cycles nanos environ
Linux 5.4 fork 97,200 31,395 [metal]
FreeBSD 12 fork 236,089 78,841 [vmware]
Darwin 20.6 fork 295,325 81,738 [metal]
NetBSD 9 fork 5,832,027 1,947,899 [vmware]
OpenBSD 6.8 fork 13,241,940 4,422,103 [vmware]
Windows10 fork 18,802,239 6,360,271 [metal]
One of the benefits of using fork() is it creates an isolation barrier between the different parts of your app. This can lead to enhanced reliability and security. For example, redbean uses fork so it can wipe your ssl keys from memory before handing over control to request handlers that process untrusted input. It also ensures that if your Lua app crashes, it won't take down the server as a whole. Hence it should come as no surprise that fork() would go slower on operating systems that have more security features. So depending on your use case, you can choose the operating system that suits you.
$PATH lookup of executable.
unix = require 'unix' prog = assert(unix.commandv('ls')) unix.execve(prog, {prog, '-hal', '.'}, {'PATH=/bin'}) unix.exit(127)
We automatically suffix .com and .exe for
all platforms when path searching. By default, the current directory
is not on the path. If prog is an absolute path, then
it's returned as-is. If prog contains slashes then it's
not path searched either and will be returned if it exists.
prog needs to be an absolute path,
see commandv(). env
defaults to to the
current environ(). Here's a
basic usage example:
unix.execve('/bin/ls', {'/bin/ls', '-hal'}, {'PATH=/bin'}) unix.exit(127)
prog needs to be the resolved pathname of your
executable. You can use commandv() to
search your PATH.
args is a string list table. The first element in args
should be prog. Values are coerced to strings. This parameter
defaults to {prog}.
env is a string list table. Values are coerced to
strings. No ordering requirement is imposed. By convention, each
string has its key and value separated by an equals sign without
spaces. If this parameter is not specified, it'll default to the
C/C++ environ variable which is inherited from the
shell that launched redbean. It's the responsibility of the user to
supply a sanitized environ when spawning untrusted processes.
execve() is normally called after fork() returns 0. If that isn't the case, then your redbean worker will be destroyed.
This function never returns on success.
unix.EAGAIN is returned if
you've enforced a max number of processes using
setrlimit(unix.RLIMIT_NPROC).
newfd may be specified to choose a specific number for
the new file descriptor. If it's already open, then the preexisting
one will be silently closed. EINVAL is returned
if newfd equals
oldfd.
flags can have unix.O_CLOEXEC which means
the returned file descriptors will be automatically closed upon
execve().
lowest defaults to zero and defines the lowest numbered file
descriptor that's acceptable to use. If newfd is specified then
lowest is ignored. For example, if you wanted to duplicate
standard input, then:
stdin2 = assert(unix.dup(0, nil, unix.O_CLOEXEC, 3))
Will ensure that, in the rare event standard output or standard error are closed, you won't accidentally duplicate standard input to those numbers.
flags can have any of
O_CLOEXEC: Automatically close file descriptor
upon execve()
O_NONBLOCK:
Request EAGAIN be
raised rather than blocking
O_DIRECT: Enable packet mode w/ atomic reads and
writes, so long as they're no larger
than PIPE_BUF
(guaranteed to be 512+ bytes) with support limited to Linux,
Windows NT, FreeBSD, and NetBSD.
Returns two file descriptors: one for reading and one for writing.
Here's an example of how pipe(), fork(), dup(), etc. may be used to serve an HTTP response containing the output of a subprocess.
local unix = require 'unix' ls = assert(unix.commandv('ls')) reader, writer = assert(unix.pipe()) if assert(unix.fork()) == 0 then unix.close(1) unix.dup(writer) unix.close(writer) unix.close(reader) unix.execve(ls, {ls, '-Shal'}) unix.exit(127) else unix.close(writer) SetHeader('Content-Type', 'text/plain') while true do data, err = unix.read(reader) if data then if data ~= '' then Write(data) else break end elseif err:errno() ~= EINTR then Log(kLogWarn, tostring(err)) break end end assert(unix.close(reader)) assert(unix.wait()) end
pid defaults to -1 which means any child process. Setting
pid to 0 is equivalent to -getpid(). If pid < -1 then
that means wait for any pid in the process group -pid. Then
lastly if pid > 0 then this waits for a specific process id
Options may have WNOHANG which means don't block, check for
the existence of processes that are already dead (technically
speaking zombies) and if so harvest them immediately.
Returns the process id of the child that terminated. In other
cases, the returned pid is nil and errno is non-nil.
The returned wstatus contains information about the process
exit status. It's a complicated integer and there's functions
that can help interpret it. For example:
-- wait for zombies -- traditional technique for SIGCHLD handlers while true do pid, status = unix.wait(-1, unix.WNOHANG) if pid then if unix.WIFEXITED(status) then print('child', pid, 'exited with', unix.WEXITSTATUS(status)) elseif unix.WIFSIGNALED(status) then print('child', pid, 'crashed with', unix.strsignal(unix.WTERMSIG(status))) end elseif status:errno() == unix.ECHILD then Log(kLogDebug, 'no more zombies') break else Log(kLogWarn, tostring(status)) break end end
exit()
assuming WIFEXITED(wstatus) is true.
WIFSIGNALED(wstatus) is true.
This function does not fail.
This function does not fail.
The impact of this action can be terminating the process, or interrupting it to request something happen.
pid can be:
pid > 0 signals one process by id
== 0 signals all processes in current process group
-1 signals all processes possible (except init)
< -1 signals all processes in -pid process group
sig can be:
0 checks both if pid exists and we can signal it
SIGINT sends ctrl-c keyboard interrupt
SIGQUIT sends backtrace and exit signal
SIGTERM sends shutdown signal
Windows NT only supports the kill() signals required by the ANSI C89
standard, which are SIGINT and SIGQUIT. All other signals on the
Windows platform that are sent to another process via kill() will be
treated like SIGKILL.
This is pretty much the same as kill(getpid(), sig).
how can be R_OK, W_OK, X_OK, or F_OK to check for
read, write, execute, and existence respectively.
flags may have any of:
AT_SYMLINK_NOFOLLOW: do not follow symbolic links.
path is the path of the directory you wish to create.
mode is octal permission bits, e.g. 0755.
Fails with EEXIST if path already exists, whether it be a
directory or a file.
Fails with ENOENT if the parent directory of the directory you
want to create doesn't exist. For making a/really/long/path/
consider using makedirs() instead.
Fails with ENOTDIR if a parent directory component existed that
wasn't a directory.
Fails with EACCES if the
parent directory doesn't grant write permission to the current user.
Fails with ENAMETOOLONG if the path is too long.
Unlike mkdir() this convenience wrapper will automatically create parent parent directories as needed. If the directory already exists then, unlike mkdir() which returns EEXIST, the makedirs() function will return success.
path is the path of the directory you wish to create.
mode is octal permission bits, e.g. 0755.
path.
path.
If path refers to a symbolic link, the link is removed.
Returns EISDIR
if path refers to a directory. See rmdir().
path.
Returns ENOTDIR
if path isn't a directory, or a path component
in path exists yet wasn't a directory.
On Windows NT a symbolic link is called a "reparse point" and can only be created from an administrator account. Your redbean will automatically request the appropriate permissions.
Note that broken links are supported on all platforms. A symbolic
link can contain just about anything. It's important to not assume
that content will be a valid filename.
On Windows NT, this function transliterates \ to / and
furthermore prefixes //?/ to WIN32 DOS-style absolute paths,
thereby assisting with simple absolute filename checks in addition
to enabling one to exceed the traditional 260 character limit.
Like unix.makedirs() this function isn't actually a system call but
rather is a Libc convenience wrapper. It's intended to be equivalent
to using the UNIX shell's rm -rf path command.
path is the file or directory path you wish to destroy.
. and .. components
removed, and symlinks will be resolved.
path is a string with the name of the file.
The asecs and ananos parameters set the access time. If they're
none or nil, the current time will be used.
The msecs and mnanos parameters set the modified time. If
they're none or nil, the current time will be used.
The nanosecond parameters (ananos and mnanos) must be on the
interval [0,1000000000) or unix.EINVAL is raised. On XNU this is
truncated to microsecond precision. On Windows NT, it's truncated to
hectonanosecond precision. These nanosecond parameters may also be
set to one of the following special values:
unix.UTIME_NOW: Fill this timestamp with current time. This
feature is not available on old versions of Linux, e.g. RHEL5.
unix.UTIME_OMIT: Do not alter this timestamp. This feature is
not available on old versions of Linux, e.g. RHEL5.
dirfd is a file descriptor integer opened with O_DIRECTORY
that's used for relative path names. It defaults to unix.AT_FDCWD.
flags may have have any of the following flags bitwise or'd
AT_SYMLINK_NOFOLLOW: Do not follow symbolic links. This makes it
possible to edit the timestamps on the symbolic link itself,
rather than the file it points to.
fd is the file descriptor of a file opened with unix.open.
The asecs and ananos parameters set the access time. If they're
none or nil, the current time will be used.
The msecs and mnanos parameters set the modified time. If
they're none or nil, the current time will be used.
The nanosecond parameters (ananos and mnanos) must be on the
interval [0,1000000000) or unix.EINVAL is raised. On XNU this is
truncated to microsecond precision. On Windows NT, it's truncated to
hectonanosecond precision. These nanosecond parameters may also be
set to one of the following special values:
unix.UTIME_NOW: Fill this timestamp with current time.
unix.UTIME_OMIT: Do not alter this timestamp.
This system call is currently not available on very old versions of Linux, e.g. RHEL5.
Returns ENOSYS on Windows NT.
On Windows NT the chmod system call only changes the read-only status of a file.
On Windows NT, this function transliterates \ to / and
furthermore prefixes //?/ to WIN32 DOS-style absolute paths,
thereby assisting with simple absolute filename checks in addition
to enabling one to exceed the traditional 260 character limit.
The returned flags may include any of:
unix.FD_CLOEXEC if fd was opened with unix.O_CLOEXEC.
Returns EBADF if fd isn't open.
flags may include any of:
unix.FD_CLOEXEC to re-open fd with unix.O_CLOEXEC.
Returns EBADF if fd isn't open.
flags & unix.O_ACCMODE includes one of:
O_RDONLY
O_WRONLY
O_RDWR
Examples of values flags & ~unix.O_ACCMODE may include:
O_NONBLOCK
O_APPEND
O_SYNC
O_ASYNC
O_NOATIME on Linux
O_RANDOM on Windows
O_SEQUENTIAL on Windows
O_DIRECT on Linux/FreeBSD/NetBSD/Windows
Examples of values flags & ~unix.O_ACCMODE won't include:
O_CREAT
O_TRUNC
O_EXCL
O_NOCTTY
Returns EBADF if fd isn't open.
Examples of values flags may include:
O_NONBLOCK
O_APPEND
O_SYNC
O_ASYNC
O_NOATIME on Linux
O_RANDOM on Windows
O_SEQUENTIAL on Windows
O_DIRECT on Linux/FreeBSD/NetBSD/Windows
These values should be ignored:
O_RDONLY, O_WRONLY, O_RDWR
O_CREAT, O_TRUNC, O_EXCL
O_NOCTTY
Returns EBADF if fd isn't open.
POSIX Advisory Locks allow multiple processes to leave voluntary hints to each other about which portions of a file they're using.
The command may be:
F_SETLK to acquire lock if possible
F_SETLKW to wait for lock if necessary
fd is file descriptor of open() file.
type may be one of:
F_RDLCK for read lock (default)
F_WRLCK for read/write lock
F_UNLCK to unlock
start is 0-indexed byte offset into file. The default is zero.
len is byte length of interval. Zero is the default and it means
until the end of the file.
whence may be one of:
SEEK_SET start from beginning (default)
SEEK_CUR start from current position
SEEK_END start from end
Returns EAGAIN if lock couldn't be acquired. POSIX says this
theoretically could also be EACCES but we haven't seen this
behavior on any of our supported platforms.
Returns EBADF if fd wasn't open.
This function accepts the same parameters as fcntl(F_SETLK) and tells you if the lock acquisition would be successful for a given range of bytes. If locking would have succeeded, then F_UNLCK is returned. If the lock would not have succeeded, then information about a conflicting lock is returned.
Returned type may be F_RDLCK or F_WRLCK.
Returned pid is the process id of the current lock owner.
This function is currently not supported on Windows.
Returns EBADF if fd wasn't open.
setpgid(0,0).
This function can be used to create daemons.
Fails with ENOSYS on Windows NT.
On Windows this system call is polyfilled by running GetUserNameW() through Knuth's multiplicative hash.
This function does not fail.
On Windows this system call is polyfilled as getuid().
This function does not fail.
For example, if your redbean is a setuid binary, then getuid() will return the uid of the user running the program, and geteuid() shall return zero which means root, assuming that's the file owning user.
On Windows this system call is polyfilled as getuid().
This function does not fail.
On Windows this system call is polyfilled as getuid().
This function does not fail.
Returns ENOSYS on Windows NT.
One use case for this function is dropping root privileges. Should
you ever choose to run redbean as root and decide not to use the
-G and -U flags, you can replicate that behavior in the Lua
processes you spawn as follows:
ok, err = unix.setgid(1000) -- check your /etc/groups if not ok then Log(kLogFatal, tostring(err)) end ok, err = unix.setuid(1000) -- check your /etc/passwd if not ok then Log(kLogFatal, tostring(err)) end
If your goal is to relinquish privileges because redbean is a setuid binary, then things are more straightforward:
ok, err = unix.setgid(unix.getgid()) if not ok then Log(kLogFatal, tostring(err)) end ok, err = unix.setuid(unix.getuid()) if not ok then Log(kLogFatal, tostring(err)) end
See also the setresuid() function and be sure to refer to your local system manual about the subtleties of changing user id in a way that isn't restorable.
If any of the above parameters are -1, then it's a no-op.
Returns ENOSYS on Windows NT.
Returns ENOSYS on Macintosh and NetBSD if saved isn't -1.
If any of the above parameters are -1, then it's a no-op.
Returns ENOSYS on Windows NT.
Returns ENOSYS on Macintosh and NetBSD if saved isn't -1.
This is used to remove bits from the mode parameter of
functions like open()
and mkdir(). The masks typically used are
027 and 022. Those masks ensure that, even if a file is created with
0666 bits, it'll be turned into 0640 or 0644 so that users other
than the owner can't modify it.
To read the mask without changing it, try doing this:
mask = unix.umask(027) unix.umask(mask)
On Windows NT this is a no-op and mask is returned.
This function does not fail.
priority is a bitmask containing the facility value and the level
value. If no facility value is ORed into priority, then the default
value set by openlog() is used. If set to NULL, the program name is
used. Level is one of LOG_EMERG, LOG_ALERT, LOG_CRIT,
LOG_ERR, LOG_WARNING, LOG_NOTICE, LOG_INFO, LOG_DEBUG.
This function currently works on Linux, Windows, and NetBSD. On WIN32 it uses the ReportEvent() facility.
>: unix.clock_gettime() 1651137352 774458779 >: Benchmark(unix.clock_gettime) 126 393 571 1
clock can be any one of of:
CLOCK_REALTIME: universally supported
CLOCK_REALTIME_FAST: ditto but faster on freebsd
CLOCK_REALTIME_PRECISE: ditto but better on freebsd
CLOCK_REALTIME_COARSE: like CLOCK_REALTIME_FAST but needs Linux 2.6.32+
CLOCK_MONOTONIC: universally supported
CLOCK_MONOTONIC_PRECISE: ditto but faster on freebsd
CLOCK_MONOTONIC_COARSE: like CLOCK_MONOTONIC_FAST but needs Linux 2.6.32+
CLOCK_MONOTONIC_RAW: actually monotonic but needs Linux 2.6.28+
CLOCK_PROCESS_CPUTIME_ID: linux and bsd
CLOCK_THREAD_CPUTIME_ID: linux and bsd
CLOCK_PROF: linux and netbsd
CLOCK_BOOTTIME: linux and openbsd
CLOCK_REALTIME_ALARM: linux-only
CLOCK_BOOTTIME_ALARM: linux-only
CLOCK_TAI: linux-only
Returns EINVAL if clock isn't supported on platform.
This function only fails if clock is invalid.
This function goes fastest on Linux and Windows.
Returns EINTR if a signal was received while waiting.
whence can be one of:
SEEK_SET: Sets the file position to offset
SEEK_CUR: Sets the file position to position + offset
SEEK_END: Sets the file position to filesize + offset
Returns the new position relative to the start of the file.
length defaults to zero.
length defaults to zero.
family defaults to AF_INET and can be:
AF_UNIX
AF_INET
type defaults to SOCK_STREAM and can be:
SOCK_STREAM
SOCK_DGRAM
SOCK_RAW
SOCK_RDM
SOCK_SEQPACKET
You may bitwise or any of the following into type:
SOCK_CLOEXEC
SOCK_NONBLOCK
protocol defaults to IPPROTO_TCP and can be:
IPPROTO_IP
IPPROTO_ICMP
IPPROTO_TCP
IPPROTO_UDP
IPPROTO_RAW
family defaults to AF_UNIX.
type defaults to SOCK_STREAM and can be:
SOCK_STREAM
SOCK_DGRAM
SOCK_SEQPACKET
You may bitwise or any of the following into type:
SOCK_CLOEXEC
SOCK_NONBLOCK
protocol defaults to 0.
ip and port are in host endian order. For example, if you
wanted to listen on 1.2.3.4:31337 you could do any of these
unix.bind(sock, 0x01020304, 31337)
unix.bind(sock, ParseIp('1.2.3.4'), 31337)
unix.bind(sock, 1 << 24 | 0 << 16 | 0 << 8 | 1, 31337)
ip and port both default to zero. The meaning of bind(0, 0)
is to listen on all interfaces with a kernel-assigned ephemeral
port number, that can be retrieved and used as follows:
sock = assert(unix.socket()) -- create ipv4 tcp socket assert(unix.bind(sock)) -- all interfaces ephemeral port ip, port = assert(unix.getsockname(sock)) print('listening on ip', FormatIp(ip), 'port', port) assert(unix.listen(sock)) assert(unix.accept(sock)) while true do client, clientip, clientport = assert(unix.accept(sock)) print('got client ip', FormatIp(clientip), 'port', clientport) unix.close(client) end
Further note that calling unix.bind(sock) is equivalent to not
calling bind() at all, since the above behavior is the default.
Tunes networking parameters.
level and optname may be one of the
following pairs. The ellipses type signature above changes depending
on which options are used.
optname is the option feature magic number. The constants for
these will be set to 0 if the option isn't supported on the host
platform.
Raises ENOPROTOOPT if your level / optname combination isn't
valid, recognized, or supported on the host platform.
Raises ENOTSOCK if fd is valid but isn't a socket.
Raises EBADF if fd isn't valid.
unix.getsockopt(fd:int, level:int, optname:int)
├─→ value:int
└─→ nil, unix.Errno
unix.setsockopt(fd:int, level:int, optname:int, value:bool)
├─→ true
└─→ nil, unix.Errno
SOL_SOCKET, SO_TYPE
SOL_SOCKET, SO_DEBUG
SOL_SOCKET, SO_ACCEPTCONN
SOL_SOCKET, SO_BROADCAST
SOL_SOCKET, SO_REUSEADDR
SOL_SOCKET, SO_REUSEPORT
SOL_SOCKET, SO_KEEPALIVE
SOL_SOCKET, SO_DONTROUTE
SOL_TCP, TCP_NODELAY
SOL_TCP, TCP_CORK
SOL_TCP, TCP_QUICKACK
SOL_TCP, TCP_FASTOPEN_CONNECT
SOL_TCP, TCP_DEFER_ACCEPT
SOL_IP, IP_HDRINCL
unix.getsockopt(fd:int, level:int, optname:int)
├─→ value:int
└─→ nil, unix.Errno
unix.setsockopt(fd:int, level:int, optname:int, value:int)
├─→ true
└─→ nil, unix.Errno
SOL_SOCKET, SO_SNDBUF
SOL_SOCKET, SO_RCVBUF
SOL_SOCKET, SO_RCVLOWAT
SOL_SOCKET, SO_SNDLOWAT
SOL_TCP, TCP_KEEPIDLE
SOL_TCP, TCP_KEEPINTVL
SOL_TCP, TCP_FASTOPEN
SOL_TCP, TCP_KEEPCNT
SOL_TCP, TCP_MAXSEG
SOL_TCP, TCP_SYNCNT
SOL_TCP, TCP_NOTSENT_LOWAT
SOL_TCP, TCP_WINDOW_CLAMP
SOL_IP, IP_TOS
SOL_IP, IP_MTU
SOL_IP, IP_TTL
unix.getsockopt(fd:int, level:int, optname:int)
├─→ secs:int, nsecs:int
└─→ nil, unix.Errno
unix.setsockopt(fd:int, level:int, optname:int, secs:int[, nanos:int])
├─→ true
└─→ nil, unix.Errno
SOL_SOCKET, SO_RCVTIMEO: If this option
is specified then your stream socket will have
a read()
/ recv() timeout. If the specified
interval elapses without receiving data,
then EAGAIN shall be
returned by read. If this option is used on listening sockets,
it'll be inherited by accepted sockets. Your redbean already does
this for GetClientFd() based on
the -t flag.
SOL_SOCKET, SO_SNDTIMEO: This is the same
as SO_RCVTIMEO but it applies to the write() / send()
functions.
unix.getsockopt(fd:int, unix.SOL_SOCKET, unix.SO_LINGER)
├─→ seconds:int, enabled:bool
└─→ nil, unix.Errno
unix.setsockopt(fd:int, unix.SOL_SOCKET, unix.SO_LINGER, secs:int, enabled:bool)
├─→ true
└─→ nil, unix.Errno
This SO_LINGER parameter can be used to make close() a blocking
call. Normally when the kernel returns immediately when it receives
close(). Sometimes it's desirable to have extra assurance on errors
happened, even if it comes at the cost of performance.
unix.setsockopt(serverfd:int, unix.SOL_TCP, unix.TCP_SAVE_SYN, enabled:int)
├─→ true
└─→ nil, unix.Errno
unix.getsockopt(clientfd:int, unix.SOL_TCP, unix.TCP_SAVED_SYN)
├─→ syn_packet_bytes:str
└─→ nil, unix.Errno
This TCP_SAVED_SYN option may be used to retrieve the bytes of the
TCP SYN packet that the client sent when the connection for fd was
opened. In order for this to work, TCP_SAVE_SYN must have been set
earlier on the listening socket. This is Linux-only. You can use the
OnServerListen hook to enable SYN saving in your Redbean. When the
TCP_SAVE_SYN option isn't used, this may return empty string.
The table of file descriptors to poll uses sparse integer keys. Any pairs with non-integer keys will be ignored. Pairs with negative keys are ignored by poll(). The returned table will be a subset of the supplied file descriptors.
events and revents may be any combination
(using bitwise OR) of:
POLLIN (events, revents): There is data to read.
POLLOUT (events, revents): Writing is now possible, although may
still block if available space in a socket or pipe is exceeded
(unless O_NONBLOCK is set).
POLLPRI (events, revents): There is some exceptional condition
(for example, out-of-band data on a TCP socket).
POLLRDHUP (events, revents): Stream socket peer closed
connection, or shut down writing half of connection.
POLLERR (revents): Some error condition.
POLLHUP (revents): Hang up. When reading from a channel such as
a pipe or a stream socket, this event merely indicates that the
peer closed its end of the channel.
POLLNVAL (revents): Invalid request.
timeoutms is the number of milliseconds to block. The default is
-1 which means block indefinitely until there's an event or an
interrupt. If the timeout elapses without any such events, an empty
table is returned. A timeout of zero means non-blocking.
mask serves the purpose of enabling poll to listen for
both file descriptor events and signals. It's equivalent to saying:
oldmask = unix.sigprocmask(unix.SIG_SETMASK, mask); unix.poll(fds, timeout); unix.sigprocmask(unix.SIG_SETMASK, oldmask);
Except it'll happen atomically on supported platforms. The only exceptions are MacOS and NetBSD where this behavior is simulated by the polyfill. Atomicity is helpful for unit testing signal behavior.
EINTR is returned if the kernel decided to deliver a
signal to a signal handler instead during your call. This is a
@norestart system call that always returns EINTR even
if SA_RESTART is in play.
flags can have any of:
SOCK_CLOEXEC
SOCK_NONBLOCK
With TCP this is a blocking operation. For a UDP socket it simply remembers the intended address so that send() or write() may be used rather than sendto().
flags can have:
MSG_WAITALL
MSG_DONTROUTE
MSG_PEEK
MSG_OOB
flags can have:
MSG_WAITALL
MSG_DONTROUTE
MSG_PEEK
MSG_OOB
write
except it has a flags argument that's intended for
sockets.
flags may have any of:
MSG_OOB
MSG_DONTROUTE
MSG_NOSIGNAL
flags may have any of:
MSG_OOB
MSG_DONTROUTE
MSG_NOSIGNAL
how is set to one of:
SHUT_RD: sends a tcp half close for reading
SHUT_WR: sends a tcp half close for writing
SHUT_RDWR
This system call currently has issues on Macintosh, so portable code should log rather than assert failures reported by shutdown().
how can be one of:
SIG_BLOCK: bitwise ors mask into set of blocked signals
SIG_UNBLOCK: removes bits in mask from set of blocked signals
SIG_SETMASK: replaces process signal mask with mask
mask is a unix.Sigset() object (see section below).
For example, to temporarily block SIGTERM and SIGINT so critical
work won't be interrupted, sigprocmask() can be used as follows:
newmask = unix.Sigset(unix.SIGTERM) oldmask = assert(unix.sigprocmask(unix.SIG_BLOCK, newmask)) -- do something... assert(unix.sigprocmask(unix.SIG_SETMASK, oldmask))
sig can be one of:
handler can be:
unix.SIG_IGN
unix.SIG_DFL
flags can have:
unix.SA_RESTART: Enables BSD signal handling semantics. Normally
i/o entrypoints check for pending signals to deliver. If one gets
delivered during an i/o call, the normal behavior is to cancel the
i/o operation and return -1 with EINTR in errno. If you use the
SA_RESTART flag then that behavior changes, so that
any function that's been annotated with @restartable will not
return EINTR and will
instead resume the i/o operation. This makes coding easier but it
can be an anti-pattern if not used carefully, since poor usage can
easily result in latency issues. It also requires one to do more
work in signal handlers, so special care needs to be given to
which C library functions are @asyncsignalsafe.
unix.SA_RESETHAND: Causes signal handler to be single-shot. This
means that, upon entry of delivery to a signal handler, it's reset
to the SIG_DFL handler automatically. You may use the alias
SA_ONESHOT for this flag, which means the same thing.
unix.SA_NODEFER: Disables the reentrancy safety check on your signal
handler. Normally that's a good thing, since for instance if your
SIGSEGV signal handler happens to segfault, you're going to want
your process to just crash rather than looping endlessly. But in
some cases it's desirable to use SA_NODEFER instead, such as at
times when you wish to longjmp() out of your signal handler and
back into your program. This is only safe to do across platforms
for non-crashing signals such as SIGCHLD and SIGINT. Crash
handlers should use Xed instead to recover execution, because on
Windows a SIGSEGV or SIGTRAP crash handler might happen on a
separate stack and/or a separate thread. You may use the alias
SA_NOMASK for this flag, which means the same thing.
unix.SA_NOCLDWAIT: Changes SIGCHLD so the zombie is gone and
you can't call wait() anymore; similar but may still deliver the
SIGCHLD.
unix.SA_NOCLDSTOP: Lets you set SIGCHLD handler that's only
notified on exit/termination and not notified on SIGSTOP,
SIGTSTP, SIGTTIN, SIGTTOU, or SIGCONT.
Example:
function OnSigUsr1(sig) gotsigusr1 = true end gotsigusr1 = false oldmask = assert(unix.sigprocmask(unix.SIG_BLOCK, unix.Sigset(unix.SIGUSR1))) assert(unix.sigaction(unix.SIGUSR1, OnSigUsr1)) assert(unix.raise(unix.SIGUSR1)) assert(not gotsigusr1) ok, err = unix.sigsuspend(oldmask) assert(not ok) assert(err:errno() == unix.EINTR) assert(gotsigusr1) assert(unix.sigprocmask(unix.SIG_SETMASK, oldmask))
It's a good idea to not do too much work in a signal handler.
The signal mask is temporarily replaced with mask during this
system call. mask specifies which signals should be blocked.
SIGALRM signals to be generated at some point(s) in the
future. The which parameter should be ITIMER_REAL.
Here's an example of how to create a 400 ms interval timer:
ticks = 0 assert(unix.sigaction(unix.SIGALRM, function(sig) print('tick no. %d' % {ticks}) ticks = ticks + 1 end)) assert(unix.setitimer(unix.ITIMER_REAL, 0, 400e6, 0, 400e6)) while true do unix.sigsuspend() end
Here's how you'd do a single-shot timeout in 1 second:
unix.sigaction(unix.SIGALRM, MyOnSigAlrm, unix.SA_RESETHAND) unix.setitimer(unix.ITIMER_REAL, 0, 0, 1, 0)
sig code into its symbolic name.
For example:
>: unix.strsignal(9) "SIGKILL" >: unix.strsignal(unix.SIGKILL) "SIGKILL"
Please note that signal numbers are normally different across supported platforms, and the constants should be preferred.
resource may be one of:
RLIMIT_AS limits the size of the virtual address space.
This will work on all platforms. It's emulated on XNU and Windows
which means it won't propagate across execve() currently.
RLIMIT_CPU causes SIGXCPU to be sent to
the process when the soft limit on CPU time is exceeded, and the
process is destroyed when the hard limit is exceeded. It works
everywhere but Windows where it should be possible to poll
getrusage() with setitimer()
RLIMIT_FSIZE causes SIGXFSZ to sent to the
process when the soft limit on file size is exceeded and the
process is destroyed when the hard limit is exceeded. It works
everywhere but Windows
RLIMIT_NPROC limits the number of simultaneous
processes and it should work on all platforms except Windows.
Please be advised it limits the process, with respect to the
activities of the user id as a whole.
RLIMIT_NOFILE limits the number of open file
descriptors and it should work on all platforms except Windows
(TODO)
If a limit isn't supported by the host platform, it'll be set to 127. On most platforms these limits are enforced by the kernel and as such are inherited by subprocesses.
hard defaults to whatever was specified in soft.
>: unix.getrusage()
{utime={0, 53644000}, maxrss=44896, minflt=545, oublock=24, nvcsw=9}
who defaults to RUSAGE_SELF and can be any of:
RUSAGE_SELF: current process
RUSAGE_THREAD: current thread
RUSAGE_CHILDREN: not supported on Windows NT
RUSAGE_BOTH: not supported on non-Linux
See the unix.Rusage section below for details on returned fields.
This can be used to sandbox your redbean workers. It allows finer
customization compared to the -S flag.
Pledging causes most system calls to become unavailable. If a forbidden system call is used, then the process will be killed. In that case, on OpenBSD, your system log will explain which promise you need. On Linux, we report the promise to stderr, with one exception: reporting is currently not possible if you pledge exec.
Using pledge is irreversible. On Linux it causes PR_SET_NO_NEW_PRIVS to be set on your process.
By default exit and exit_group are always allowed. This is useful for processes that perform pure computation and interface with the parent via shared memory.
Once pledge is in effect, the chmod functions (if allowed) will not permit the sticky/setuid/setgid bits to change. Linux will EPERM here and OpenBSD should ignore those three bits rather than crashing.
User and group IDs also can't be changed once pledge is in effect. OpenBSD should ignore the chown functions without crashing. Linux will just EPERM.
Root access isn't required. Support is limited to OpenBSD and Linux
2.6.23+ (i.e. RHEL6 c. 2012) so long as Redbean is running directly
on the host system, i.e. not running in a userspace emulator like
Blink or Qemu. If your environment isn't supported, then pledge()
will return 0 and do nothing, rather than raising ENOSYS, so the
apps you share with others will err on the side of not breaking. If
a functionality check is needed, please use unix.pledge(nil, nil)
which is a no-op that will fail appropriately when the necessary
system support isn't available to impose security restrictions.
promises is a string that may include any of the following groups
delimited by spaces. This list has been curated to focus on the
system calls for which this module provides wrappers. See the
Cosmopolitan Libc pledge() documentation for a comprehensive and
authoritative list of raw system calls. Having the raw system call
list may be useful if you're executing foreign programs.
This may be needed to launch non-static non-native executables, such as non-assimilated APE binaries, or programs that link dynamic shared objects, i.e. most Linux distro binaries.
execpromises only matters if "exec" is specified
in promises. In that case, this specifies the promises
that'll apply once execve() happens. If this is NULL then the default
is used, which is unrestricted. OpenBSD allows child processes to
escape the sandbox (so a pledged OpenSSH server process can do things
like spawn a root shell). Linux however requires monotonically
decreasing privileges. This function will will perform some validation
on Linux to make sure that execpromises is a subset
of promises. Your libc wrapper for execve() will then
apply its SECCOMP BPF filter later. Since Linux has to do this before
calling sys_execve(), the executed process will be weakened to have
execute permissions too.
mode if specified should specify one penalty:
unix.PLEDGE_PENALTY_KILL_THREAD causes the violating
thread to be killed. This is the default on Linux. It's effectively
the same as killing the process, since redbean has no threads. The
termination signal can't be caught and will be
either SIGSYS or SIGABRT. Consider
enabling stderr logging below so you'll know why your program
failed. Otherwise check the system log.
unix.PLEDGE_PENALTY_KILL_PROCESS causes the process and
all its threads to be killed. This is always the case on OpenBSD.
unix.PLEDGE_PENALTY_RETURN_EPERM causes system calls to
just return an EPERM error instead of killing. This is
a gentler solution that allows code to display a friendly warning.
Please note this may lead to weird behaviors if the software being
sandboxed is lazy about checking error results.
mode may optionally bitwise or the following flags:
unix.PLEDGE_STDERR_LOGGING enables friendly error
message logging letting you know which promises are needed whenever
violations occur. Without this, violations will be logged to
dmesg on Linux if the penalty is to kill the process.
You would then need to manually look up the system call number and
then cross reference it with the cosmopolitan libc pledge()
documentation. You can also use strace -ff which is
easier. This is ignored OpenBSD, which already has a good system
log. Turning on stderr logging (which uses SECCOMP trapping) also
means that the unix.WTERMSIG() on your killed processes
will always be unix.SIGABRT on both Linux and OpenBSD.
Otherwise, Linux prefers to raise unix.SIGSYS.
unix.unveil(".", "r"); -- current dir + children visible unix.unveil("/etc", "r"); -- make /etc readable too unix.unveil(nil, nil); -- commit and lock policyUnveiling restricts a thread's view of the filesystem to a set of allowed paths with specific privileges.
Once you start using unveil(), the entire file system is considered
hidden. You then specify, by repeatedly calling unveil(), which paths
should become unhidden. When you're finished, you call
unveil(0,0) which commits your policy, after which further
use is forbidden, in the current thread, as well as any threads or
processes it spawns.
There are some differences between unveil() on Linux versus OpenBSD.
unveil(0,0) which commits and locks.
This system call is supported natively on OpenBSD and polyfilled on Linux using the Landlock LSM[1].
This function requires OpenBSD or Linux 5.13+ (2022+). If the kernel
support isn't available (or we're in an emulator like Qemu or Blink)
then zero is returned and nothing happens (instead of raising
ENOSYS) because the files are still unveiled.
Use unix.unveil("", nil) to feature check the host
system, which is defined as a no-op that'll fail if the host system
doesn't have the necessary features that allow unix.unveil() impose
bona-fide security restrictions. Otherwise, if everything is good, a
return value >=0 is returned, where 0
means OpenBSD, and >=1 means Linux with Landlock
LSM, in which case the return code shall be the maximum supported
Landlock ABI version.
path is the file or directory to unveil
permissions is a string consisting of zero or more of the
following characters:
r makes path available for read-only path
operations, corresponding to the pledge promise "rpath".
w makes path available for write
operations, corresponding to the pledge promise "wpath".
x makes path available for execute
operations, corresponding to the pledge promises "exec" and
"execnative".
c allows path to be created and removed,
corresponding to the pledge promise "cpath".
This function is like localtime() except it always returns Greenwich Mean Time irrespective of the TZ environment variable.
For example:
>: unix.gmtime(unix.clock_gettime()) 2022 5 11 22 43 20 0 3 130 0 "GMT"
Here's how you might format a localized timestamp with nanoseconds:
>: unixsec, nanos = unix.clock_gettime()
>: year,mon,mday,hour,min,sec = unix.localtime(unixsec)
>: '%.4d-%.2d-%.2dT%.2d:%.2d:%.2d.%.9dZ' % {year,mon,mday,hour,min,sec,nanos}
"2022-05-11T15:46:32.160239978Z"
year is the year, where zero is defined as 0 A.D. This
value may be on the interval -13.7e9 ≤ year ≤ 10e14
which is the time from the Big Bang, through most of the
Stelliferous Era.
mon is the month of the year, on the interval 1 ≤
mon ≤ 12 in order to make printf style formatting easier.
mday is the day of the month, on the interval 1 ≤
mday ≤ 31 in order to make printf style formatting easier.
hour represent hours, on the interval 0 ≤ hour ≤
23.
min represents minutes, on the interval 0 ≤ min ≤
59.
sec represents seconds, on the interval 0 ≤ sec ≤
60. Please note this is a 61 second interval in order to
accommodate highly rare leap second events.
wday is the day of the week, on the interval 0 ≤
wday ≤ 6.
yday is the day of the year on the interval 0 ≤
yday ≤ 365.
gmtoff is the Zulu time offset in seconds, which should
be on the interval ±93600 seconds.
dst will be 1 if daylight savings, 0 if not daylight
savings, or -1 if it couldn't be determined.
>: unix.localtime(unix.clock_gettime()) 2022 4 28 2 14 22 -25200 4 117 1 "PDT"
This follows the same API as gmtime() except it takes the TZ
environment variable into consideration to determine the most
appropriate localization.
Please see the gmtime() function for documentaiton on the meaning of the various returned values.
Here's an example of how you might format a localized timestamp:
>: unixsec, nanos = unix.clock_gettime()
>: year, mon, mday, hour, min, sec, gmtoffsec = unix.localtime(unixsec)
>: '%.4d-%.2d-%.2dT%.2d:%.2d:%.2d.%.9d%+.2d%.2d' % {
year, mon, mday, hour, min, sec, nanos,
gmtoffsec / (60 * 60), math.abs(gmtoffsec) % 60}
"2022-05-11T15:46:32.160239978-0700"
Your redbean ships with a subset of the time zone database.
You can control which timezone is used using the TZ environment
variable. If your time zone isn't included in the above list, you
can simply copy it inside your redbean. The same is also the case
for future updates to the database, which can be swapped out when
needed, without having to recompile.
flags may have any of:
AT_SYMLINK_NOFOLLOW: do not follow symbolic links.
dirfd defaults to to unix.AT_FDCWD and may optionally be set to
a directory file descriptor to which path is relative.
A common use for fstat() is getting the size of a file. For example:
fd = assert(unix.open('hello.txt', unix.O_RDONLY)) st = assert(unix.fstat(fd)) Log(kLogInfo, 'hello.txt is %d bytes in size' % {st:size()}) unix.close(fd)
path is the path of a file or directory in the mounted
filesystem.
fd is an open() file descriptor of a file or directory in
the mounted filesystem.
For example, to print a simple directory listing:
Write('<ul>\r\n') for name, kind, ino, off in assert(unix.opendir(dir)) do if name ~= '.' and name ~= '..' then Write('<li>%s\r\n' % {EscapeHtml(name)}) end end Write('</ul>\r\n')
fd should be created by open(path, unix.O_RDONLY|unix.O_DIRECTORY). The
returned unix.Dir takes ownership of the file descriptor and will
close it automatically when garbage collected.
ENOTTY if fd is valid but not a teletypewriter
EBADF if fd isn't a valid file descriptor.
EPERM if pledge() is used without tty in lenient mode
No other error numbers are possible.
This creates a secure temporary file inside $TMPDIR. If
it isn't defined, then /tmp is used on UNIX and
GetTempPath() is used on the New Technology. This resolution
of $TMPDIR happens once, which is copied to
the kTmpDir global.
Once close() is called, the returned file is guaranteed to be deleted automatically. On UNIX the file is unlink()'d before this function returns. On the New Technology it happens upon close().
On the New Technology, temporary files created by this function
should have better performance, because kNtFileAttributeTemporary
asks the kernel to more aggressively cache and reduce i/o ops.
This function allocates special memory that'll be inherited across fork in a shared way. By default all memory in Redbean is "private" memory that's only viewable and editable to the process that owns it. When unix.fork() happens, memory is copied appropriately so that changes to memory made in the child process, don't clobber the memory at those same addresses in the parent process. If you don't want that to happen, and you want the memory to be shared similar to how it would be shared if you were using threads, then you can use this function to achieve just that.
The memory object this function returns may be accessed using its methods, which support atomics and futexes. It's very low-level. For example, you can use it to implement scalable mutexes:
mem = unix.mapshared(8000 * 8) LOCK = 0 -- pick an arbitrary word index for lock -- From Futexes Are Tricky Version 1.1 § Mutex, Take 3; -- Ulrich Drepper, Red Hat Incorporated, June 27, 2004. function Lock() local ok, old = mem:cmpxchg(LOCK, 0, 1) if not ok then if old == 1 then old = mem:xchg(LOCK, 2) end while old > 0 do mem:wait(LOCK, 2) old = mem:xchg(LOCK, 2) end end end function Unlock() old = mem:fetch_add(LOCK, -1) if old == 2 then mem:store(LOCK, 0) mem:wake(LOCK, 1) end end
It's possible to accomplish the same thing as unix.mapshared() using files and unix.fcntl() advisory locks. However this goes significantly faster. For example, that's what SQLite does and we recommend using SQLite for IPC in redbean. But, if your app has thousands of forked processes fighting for a file lock you might need something lower level than file locks, to implement things like throttling. Shared memory is a good way to do that since there's nothing that's faster.
The size parameter needs to be a multiple of 8. The
returned memory is zero initialized. When allocating shared memory,
you should try to get as much use out of it as possible, since the
overhead of allocating a single shared mapping is 500 words of
resident memory and 8000 words of virtual memory. It's because the
Cosmopolitan Libc mmap() granularity is 2**16.
This system call does not fail. An exception is instead thrown if sufficient memory isn't available.
unix.Memory encapsulates memory that's shared across fork() and this module provides the fundamental synchronization primitives. These objects are created by unix.mapshared().
Redbean memory maps may be used in two ways:
They're aliased, union, or overlapped views of the same memory. For example if you write a string to your memory region, you'll be able to read it back as an integer.
Reads, writes, and word operations will throw an exception if a memory boundary error or overflow occurs.
offset is the starting byte index from which memory is
copied, which defaults to zero.
If bytes is none or nil, then the nul-terminated string
at `offset` is returned. You may specify bytes to
safely read binary data.
This operation happens atomically. Each shared mapping has a single lock which is used to synchronize reads and writes to that specific map. To make it scale, create additional maps.
offset is the starting byte index to which memory is
copied, which defaults to zero.
If bytes is none or nil, then an implicit
nil-terminator will be included after your data so
things like json can be easily serialized to shared memory.
This operation happens atomically. Each shared mapping has a single lock which is used to synchronize reads and writes to that specific map. To make it scale, create additional maps.
This operation is atomic and has relaxed barrier semantics.
This operation is atomic and has relaxed barrier semantics.
This sets word at word_index to value and
returns the value previously held within that word.
This operation is atomic and provides the same memory barrier semantics as the aligned x86 LOCK XCHG instruction.
This inspects the word at word_index and if its value
is the same as old then it'll be replaced by the
value new, in which case true shall be returned
alongside old. If a different value was held at word,
then false shall be returned along with its value.
This operation happens atomically and provides the same memory barrier semantics as the aligned x86 LOCK CMPXCHG instruction.
This method modifies the word at word_index to contain
the sum of its value and the value parameter. This
method then returns the value as it existed before the addition was
performed.
This operation is atomic and provides the same memory barrier semantics as the aligned x86 LOCK XADD instruction.
This operation happens atomically and provides the same memory barrier ordering semantics as its x86 implementation.
This operation happens atomically and provides the same memory barrier ordering semantics as its x86 implementation.
This operation happens atomically and provides the same memory barrier ordering semantics as its x86 implementation.
This method asks the kernel to suspend the process until either the absolute deadline expires or we're woken up by another process that calls unix.Memory:wake().
The expect parameter is the value you expect the word to
have and this function will return if that's not the case. Please
note this parameter doesn't imply the kernel will poll the value for
you, and you still need to call wake() when you know the memory's
changed.
The default behavior is to wait until the heat death of the universe if necessary. You may alternatively specify an absolute deadline. If it's less than or equal to the value returned by clock_gettime, then this routine is non-blocking. Otherwise we'll block at most until the current time reaches the absolute deadline.
Futexes are supported natively on Linux, FreeBSD, and OpenBSD. When this interface is used on other platforms this method will manually poll the memory location with exponential backoff. Doing this works well enough that we're passing the *NSYNC unit tests, but is not as low latency as having kernel supported futexes.
EINTR if a signal is delivered while waiting on
deadline. Callers should use futexes inside a loop that is able to
cope with spurious wakeups. We don't actually guarantee the value at
word has in fact changed when this returns.
EAGAIN is raised if, upon entry, the word
at word_index had a different value than what's
specified at expect.
ETIMEDOUT is raised when the absolute deadline expires.
This method may be used to signal or broadcast to waiters. The
count specifies the number of processes that should be
woken, which defaults to INT_MAX.
The return value is the number of processes that were actually woken as a result of the system call. No failure conditions are defined.
unix.Dir objects are created by opendir() or fdopendir(). The following methods are available:
This is called automatically by the garbage collector.
This may be called multiple times.
Returns nil if there are no more entries. On
error, nil will be returned and errno will
be non-nil.
kind can be any of:
DT_REG: file is a regular file
DT_DIR: file is a directory
DT_BLK: file is a block device
DT_LNK: file is a symbolic link
DT_CHR: file is a character device
DT_FIFO: file is a named pipe
DT_SOCK: file is a named socket
DT_UNKNOWN
Note: This function also serves as the __call metamethod, so that
unix.Dir objects may be used as a for loop iterator.
Returns EOPNOTSUPP if using a /zip/... path.
Returns EOPNOTSUPP if using Windows NT.
unix.Rusage objects are created by wait() or getrusage(). The following accessor methods are available.
It's always the case that 0 ≤ nanos < 1e9.
On Windows NT this is collected from GetProcessTimes().
It's always the case that 0 ≤ 𝑥 < 1e9.
On Windows NT this is collected from GetProcessTimes().
On Windows NT this is collected from NtProcessMemoryCountersEx::PeakWorkingSetSize / 1024.
If you chart memory usage over the lifetime of your process, then
this would be the space filled in beneath the chart. The frequency
of kernel scheduling is defined
as CLK_TCK. Each time a
tick happens, the kernel samples your process's memory usage, by
adding it to this value. You can derive the average consumption from
this value by computing how many ticks are in utime +
stime.
Currently only available on FreeBSD and NetBSD.
If you chart memory usage over the lifetime of your process, then
this would be the space filled in beneath the chart. The frequency
of kernel scheduling is defined
as CLK_TCK. Each time a
tick happens, the kernel samples your process's memory usage, by
adding it to this value. You can derive the average consumption from
this value by computing how many ticks are in utime +
stime.
Currently only available on FreeBSD and NetBSD.
If you chart memory usage over the lifetime of your process, then
this would be the space filled in beneath the chart. The frequency
of kernel scheduling is defined
as CLK_TCK. Each time a
tick happens, the kernel samples your process's memory usage, by
adding it to this value. You can derive the average consumption from
this value by computing how many ticks are in utime +
stime.
This is only applicable to redbean if its built with MODE=tiny, because redbean likes to allocate its own deterministic stack.
Currently only available on FreeBSD and NetBSD.
This number indicates how many times redbean was preempted by the kernel to memcpy() a 4096-byte page. This is one of the tradeoffs fork() entails. This number is usually tinier, when your binaries are tinier.
Not available on Windows NT.
This number indicates how many times redbean was preempted by the kernel to perform i/o. For example, you might have used mmap() to load a large file into memory lazily.
On Windows NT this is NtProcessMemoryCountersEx::PageFaultCount.
Operating systems like to reserve hard disk space to back their RAM guarantees, like using a gold standard for fiat currency. When your system is under heavy memory load, swap operations may happen while redbean is working. This number keeps track of them.
Not available on Linux, Windows NT.
On Windows NT this is NtIoCounters::ReadOperationCount.
On Windows NT this is NtIoCounters::WriteOperationCount.
Not available on Linux, Windows NT.
Not available on Linux, Windows NT.
Not available on Linux.
This number is a good thing. It means your redbean finished its work quickly enough within a time slice that it was able to give back the remaining time to the system.
This number is a bad thing. It means your redbean was preempted by a higher priority process after failing to finish its work, within the allotted time slice.
unix.Stat objects are created by stat() or fstat(). The following accessor methods are available.
For example, 0010644 is what you might see for a file and
0040755 is what you might see for a directory.
To determine the file type:
unix.S_ISREG(st:mode()) means fifo or pipe
unix.S_ISDIR(st:mode()) means character device
unix.S_ISLNK(st:mode()) means directory
unix.S_ISCHR(st:mode()) means block device
unix.S_ISBLK(st:mode()) means regular file
unix.S_ISFIFO(st:mode()) means symbolic link
unix.S_ISSOCK(st:mode()) means socket
This field should be accurate on Apple, Windows, and BSDs. On Linux this is the minimum of atim/mtim/ctim. On Windows NT nanos is only accurate to hectonanoseconds.
Here's an example of how you might print a file timestamp:
st = assert(unix.stat('/etc/passwd')) unixts, nanos = st:birthtim() year,mon,mday,hour,min,sec,gmtoffsec = unix.localtime(unixts) Write('%.4d-%.2d-%.2dT%.2d:%.2d:%.2d.%.9d%+.2d%.2d % { year, mon, mday, hour, min, sec, nanos, gmtoffsec / (60 * 60), math.abs(gmtoffsec) % 60})
Please note that file systems are sometimes mounted with noatime
out of concern for i/o performance. Linux also provides O_NOATIME
as an option for open().
On Windows NT this is the same as birth time.
Means time file status was last changed on UNIX.
On Windows NT this is the same as birth time.
This provides some indication of how much physical storage a file actually consumes. For example, for small file systems, your system might report this number as being 8, which means 4096 bytes.
On Windows NT, if O_COMPRESSED is used for a file, then
this number will reflect the size after compression. you
can use:
st = assert(unix.stat("moby.txt")) print('file size is %d bytes' % {st:size()}) print('file takes up %d bytes of space' % {st:blocks() * 512}) if GetHostOs() == 'WINDOWS' and st:flags() & 0x800 then print('thanks to file system compression') end
To tell if compression is used on a file.
This field might be of assistance in computing optimal i/o sizes.
Please note this field has no relationship to blocks, as the latter is fixed at a 512 byte size.
This can be used to detect some other process used rename() to swap out a file underneath you, so you can do a refresh. redbean does it during each main process heartbeat for its own use cases.
On Windows NT this is set to NtByHandleFileInformation::FileIndex.
On Windows NT this is NtByHandleFileInformation::VolumeSerialNumber.
This value may be set to 0 or -1 for files that aren't devices, depending on the operating system. unix.major() and unix.minor() may be used to extract the device numbers.
unix.Statfs objects are created by statfs() or fstatfs(). The following accessor methods are available.
Here's some examples of likely values:
"ext" on Linux
"xfs" on RHEL7
"apfs" on Apple
"zfs" on FreeBSD
"ffs" on NetBSD and OpenBSD
"NTFS" on Windows
This is a platform-specific magic number. Consider using the unix.Statfs:fstypename() method instead. On Windows, this will actually be a Knuth multiplicative hash of the name.
This field serves two purposes:
blocks, bfree,
and bavail to obtain a byte count.
The size of a block is measured as unix.Statfs:bsize().
The size of a block is measured as unix.Statfs:bsize().
The size of a block is measured as unix.Statfs:bsize().
On Windows this is always the maximum integer value.
On Windows this is always the maximum integer value.
The following flags are defined:
ST_RDONLY: Read-only filesystem (Linux/Windows/XNU/BSDs)
ST_NOSUID: Setuid binaries forbidden (Linux/XNU/BSDs)
ST_NODEV: Device files forbidden (Linux/XNU/BSDs)
ST_NOEXEC: Execution forbidden (Linux/XNU/BSDs)
ST_SYNCHRONOUS: Synchronous (Linux/XNU/BSDs)
ST_NOATIME: No access time (Linux/XNU/BSDs)
ST_RELATIME: Relative access time (Linux/NetBSD)
ST_APPEND: Linux-only
ST_IMMUTABLE: Linux-only
ST_MANDLOCK: Linux-only
ST_NODIRATIME: Linux-only
ST_WRITE: Linux-only
On Linux this is always 0 for root. On Windows this is always 0.
The unix.Sigset class defines a mutable bitset that may currently
contain 128 entries. See unix.NSIG to find out how many signals
your operating system actually supports.
sig is member of signal bitset.
This object is returned by system calls that fail. We prefer returning an object because for many system calls, an error is part their normal operation. For example, it's often desirable to use the errno() method when performing a read() to check for EINTR.
The error number is always different for different platforms. On UNIX systems, error numbers occupy the range [1,127] in practice. The System V ABI reserves numbers as high as 4095. On Windows NT, error numbers can go up to 65535.
On UNIX systems this is always 0. On Windows NT this will normally be the same as errno(). Because Windows defines so many error codes, there's oftentimes a multimapping between its error codes and System Five. In those cases, this value reflect the GetLastError() result at the time the error occurred.
For example, this might return "EINTR".
For example, this might return "read" if read() was what failed.
For example, this might return "Interrupted system call".
Different information components are delimited by slash.
For example, this might return "EINTR/4/Interrupted system call".
On Windows NT this will include additional information about the Windows error (including FormatMessage() output) if the WIN32 error differs from the System Five error code.
Raised by [pretty much everything].
On Windows this is raised by chroot, setuid, setgid, getsid, setsid, and others we're doing our best to document.
Raised by access, bind, chdir, chmod, chown, chroot, clock_getres, execve, opendir, link, mkdir, mknod, open, readlink, rename, rmdir, stat, symlink, truncate, unlink, utime, utimensat.
open("foo/bar") and foo is a regular file, then ENOTDIR will
be returned.
Raised by open, access, chdir, chroot, execve, link, mkdir, mknod, opendir, readlink, rename, rmdir, stat, symlink, truncate, unlink, utimensat, bind, chmod, chown, fcntl, futimesat.
Raised by accept, clock_nanosleep, close, connect, dup, fcntl, flock, getrandom, nanosleep, open, pause, poll, ptrace, read, recv, select, send, sigsuspend, sigwaitinfo, truncate, wait, write.
Raised by execve.
Raised by execve.
Raised by wait.
Raised by getpriority, getrlimit, getsid, ioprio_set, kill, setpgid, utimensat.
Raised by accept, access, bind, chdir, chmod, chown, close, connect, copy_file_range, dup, fcntl, flock, fsync, futimesat, opendir, getpeername, getsockname, getsockopt, ioctl, link, listen, llseek, lseek, mkdir, mknod, mmap, open, prctl, read, readahead, readlink, recv, rename, select, send, shutdown, splice, stat, symlink, sync, sync_file_range, truncate, unlink, utimensat, write.
Raised by accept, connect, fcntl, fork, getrandom, mincore, mlock, mmap, mremap, poll, read, select, send, setresuid, setreuid, setuid, sigwaitinfo, splice, tee, timer_create, kill, write,
This happens when you try to write data to a subprocess via a pipe but
the reader end has already closed, possibly because the process died.
Normally i/o routines only return this if SIGPIPE doesn't
kill the process. Unlike default UNIX programs, redbean currently
ignores SIGPIPE by default, so this error code is a
distinct possibility when pipes or sockets are being used.
PATH_MAX as
1024 characters. On UNIX that limit should only apply to system call
wrappers like realpath. On Windows NT it's observed by all system
calls that accept a pathname.
Raised by access, bind, chdir, chmod, chown, chroot, execve, gethostname, link, mkdir, mknod, open, readlink, rename, rmdir, stat, symlink, truncate, unlink, utimensat.
Raised by access, bind, chdir, chmod, chown, chroot, clock_getres, connect, execve, fcntl, getpriority, link, mkdir, mknod, mmap, mprotect, msgctl, open, prctl, ptrace, readlink, rename, rmdir, semget, send, setpgid, socket, stat, symlink, truncate, unlink, uselib, utime, utimensat.
Raised by access, bind, chdir, chmod, chown, chroot, clone, copy_file_range, execve, fork, getgroups, getrlimit, ioperm, link, mbind, mincore, mkdir, mknod, mlock, mmap, mprotect, mremap, msync, open, poll, readlink, recv, rename, rmdir, select, send, sigaltstack, splice, stat, symlink, sync_file_range, tee, unlink.
Raised by accept, chmod, chown, chroot, copy_file_range, execve, fcntl, getdomainname, gethostname, getrlimit, getsid, ioperm, iopl, kill, link, mkdir, mknod, nice, open, rename, rmdir, sched_setaffinity, sched_setscheduler, seteuid, setfsgid, setfsuid, setgid, setns, setpgid, setresuid, setreuid, setsid, setuid, setup, setxattr, sigaltstack, spu_create, stime, symlink, syslog, truncate, unlink, utime, utimensat, write.
Raised by umount.
Raised by link, mkdir, mknod, mmap, open, rename, rmdir, symlink.
Raised by copy_file_range, link, rename.
Raised by copy_file_range, execve, open, read, rename, truncate, unlink.
Raised by accept, execve, mmap, open, pipe, socket, socketpair.
Raised by accept, dup, execve, fcntl, open, pipe, socket, socketpair.
Raised by ioctl.
Raised by access, copy_file_range, execve, mmap, open, truncate.
Raised by copy_file_range, open, truncate, write.
Raised by copy_file_range, fsync, link, mkdir, mknod, open, rename, symlink, sync_file_range, write.
Raised by lseek, splice, sync_file_range.
Raised by access, bind, chmod, chown, link, mkdir, mknod, open, rename, rmdir, symlink, truncate, unlink, utime, utimensat.
Raised by prctl.
Raised by fcntl.
Raised by rmdir.
Raised by access, bind, chdir, chmod, chown, chroot, execve, link, mkdir, mknod, open, readlink, rename, rmdir, stat, symlink, truncate, unlink, utimensat.
Raised by connect.
Raised by accept, bind, connect, getpeername, getsockname, getsockopt, listen, recv, send, shutdown.
Raised by send.
Raised by connect.
Raised by getsockopt, accept.
Raised by socket, socketpair.
Raised by chmod, clock_getres, clock_nanosleep, timer_create.
Raised by accept, listen, mmap, prctl, readv, send, socketpair.
Raised by connect, socket, socketpair.
Raised by accept.
Raised by accept.
Raised by send.
Raised by getpeername, getsockname, send.
Raised by getpeername, recv, send, shutdown.
EPIPE.
Raised by sendmsg.
Raised by connect.
Raised by accept.
Raised by accept.
This is the character limit when
calling execve(). It's the sum of the
lengths of argv and envp including any nul
terminators and pointer arrays. For example to see how much your
shell envp uses
$ echo $(($(env | wc -c) + 1 + ($(env | wc -l) + 1) * 8)) 758
POSIX mandates this be 4096 or higher. On Linux this it's 128*1024. On Windows NT it's 32767*2 because CreateProcess lpCommandLine and environment block are separately constrained to 32,767 characters. Most other systems define this limit much higher.
The UNIX module does not perform any buffering between calls.
Each time a read or write is performed via the UNIX API your redbean will allocate a buffer of this size by default. This current default would be 4096 across platforms.
This is granularity at which the kernel does work. For example, the Linux kernel normally operates at 100hz so its CLK_TCK will be 100.
This value is useful for making sense out of unix.Rusage data.
POSIX requires this be at least 512. Linux is more generous and allows 4096. On Windows NT this is currently 4096, and it's the parameter redbean passes to CreateNamedPipe().
This applies to a complete path being passed to system calls.
POSIX.1 XSI requires this be at least 1024 so that's what most
platforms support. On Windows NT, the limit is technically 260
characters. Your redbean works around that by prefixing //?/
to your paths as needed. On Linux this limit will be 4096, but
that won't be the case for functions such as realpath that are
implemented at the C library level; however such functions are
the exception rather than the norm, and report enametoolong(),
when exceeding the libc limit.
POSIX requires this be at least 14. Most operating systems define it as 255. It's a good idea to not exceed 253 since that's the limit on DNS labels.
The limit for unix.Sigset is 128 to support FreeBSD, but most operating systems define this much lower, like 32. This constant reflects the value chosen by the underlying operating system.
Log.
kLogDebug. See Log.
kLogVerbose. See Log.
kLogVerbose. See Log.
kLogWarn. See Log.
kLogError.
See Log. Logging anything at this
level will result in a backtrace and process exit.
You can have redbean run as a daemon by doing the following:
redbean.com -vv -d -L redbean.log -P redbean.pid kill -TERM $(cat redbean.pid) # 1x: graceful shutdown kill -TERM $(cat redbean.pid) # 2x: forceful shutdown
It's possible to modify global interpreter state later on in the
server's lifecycle. When running in daemon mode, using kill -HUP
$(pidof redbean.com) will instruct redbean to run the code
in .reload.lua from the main process,
will will be lazily propagated to client connections.
You can modify the zip while redbean is running. The zip command by
default will do this by replacing the inode. redbean will detect the
changed inode within a second and broadcast SIGUSR1 to the process group
so the new assets get indexed as soon as possible. It's also possible to
modify the executable assets in place, while the executable is running.
If you do that, then you need to be careful to not disturb the local
file and central directory. What you would do instead is append changed
or new files. Then append a new central directory, along with a new end
of central directory record. Finally, memset(0) the old end of central
directory record. redbean will detect it's gone and reindex. You can
even modify local files in place too. The way you would do that is by
clearing the PK♥♦ magic marker while the file memory is
being mutated, and then putting it back. Any requests that arrive during
the modification will result in a 503 Service Unavailable so your load
balancer can failover.
redbean will grow to whatever number of processes your system limits and
tcp stack configuration allow. Once functions like fork()
and accept() start to fail, redbean will enter "meltdown
mode" where it interrupts worker processes to immediately close idling
and lagging connections. redbean may need to drop requests by sending
503 Service Unavailable until congestion subsides. So be
sure your load balancer is configured to immediately failover to another
instance in such cases.
There's a 64kb limit on request message size, where the header portion is further limited to 32kb. We do that to guarantee processes stay tiny. You can tune it using the -M flag. redbean spawns a process for each connection. redbean needs about 200kb of RAM per worker on average. Clients are encouraged to pipeline HTTP requests within the same connection.
redbean rejects requests for hidden files, i.e. any path containing the
substring /. and requests with denormalized paths,
e.g. /../../etc/passwd are also categorically rejected.
Furthermore, redbean won't service requests that come in more than 32
fragments. Those few restrictions aside, redbean generally aims to
follow Postel's maxim in the sense that it's liberal in what it accepts
but conservative in what it sends.
If you want Rust-like promises then redbean can be compiled with ASAN memory safety. Cosmopolitan Libc has the only open source implementation of the Address Sanitizer runtime that's intended for production use. It causes redbean worker processes to crash and log a report should a bug like a buffer overrun, use-after-free, or stack smash occur. This has a marginal impact on performance. It can be useful in environments where interruptions in a service are more desirable than risking the system being compromised.
redbean doesn't secure your computer. It does however provide tools we hope will help you do it yourself. For further details on why things need to be this way, please see the disclaimer in the ISC license.
Some computing environments rely on physical security and redbean is a good fit for that. For example, if you need to run a web app on an air-gapped computer running an old version of some other operating system that can't be upgraded, then you load your redbean off a thumb drive provided that the system was installed after the year 2007 or more specifically runs x86_64 with Linux 2.6.18+ or Windows Vista+.
Some environments require that security be provided using existing infrastructure like SSL frontends. In that case, the "redbean-unsecure" download link might be the right choice for you, since it's a special build of redbean that leaves out the security code. That way, you can bolt the security on separately using a tool like stunnel.
redbean provides integrated SSL support based on MbedTLS. It's configured to offer 128 bits of security with modern clients, but will fall back to at minimum 112 bits of security depending on the preferences of the client. Both are secure based on public knowledge until 2030 according to NIST. If you'd rather restrict yourself to just 150+ bits of security but with the tradeoff of dropping support for old Internet Explorer and making embedded clients less happy, then pass the -B flag, which'll restrict redbean to a very short list of protocols, algorithms, and parameters that the NSA, NIST, and IANA all agree upon.
redbean's SSL implementation is tuned for performance. It uses hardware algorithms when available such as AES-NI, SHA-NI, and RDRAND. redbean does not use costly hardnening measures specific only to legacy clients like Internet Explorer if they increase denial of service risk for the server as a whole.
redbean is tuned for ease of use. Your redbean uses a protocol polyglot for serving HTTP and HTTPS on the same port numbers. Both the TLS hello and the SSLv2 hello are accepted, even though only TLS is supported. For example, both of these are valid:
http://127.0.0.1:8080/ https://127.0.0.1:8080/
By default, your redbean will automatically generate ephemeral self-signed ECDSA and RSA serving certificates. This causes browser warnings. The simplest option for making the warning go away is to give redbean a key signing key (KSK).
openssl req -x509 -newkey rsa:2048 \ -keyout .ca.key -out .ca.crt -days 6570 -nodes \ -subj '/C=US/ST=CA/O=Jane Doe/CN=My Root CA 1' \ -addext 'keyUsage = critical,cRLSign,keyCertSign' sudo ./redbean.com -C ca.crt -K .ca.key -p 80 -p 443
Your SSL root can then be installed on client machines as follows:
# linux sudo cp ca.crt /usr/local/share/ca-certificates sudo update-ca-certificates # macos sudo security add-trusted-cert -d -r trustRoot \ -k /Library/Keychains/System.keychain ca.crt # windows certutil -addstore -f "ROOT" ca.crt # notes # firefox is special you have to use its settings
If your goal is to make SSL deploys easy, then it's possible to put the
KSK inside the redbean.com file using the InfoZIP program. Be sure the
key is a hidden file. It can be loaded using your .init.lua
script with the LoadAsset,
ProgramCertificate, and
ProgramPrivateKey APIs.
Please note, this is just an example of what you could do; we don't
claim it's what you should do.
For a public-facing online service, the simplest way to use SSL is with Let's Encrypt. Let's say you're migrating from NGINX. In that case you'll likely want something like the following:
# commands subject to public monitoring
certbot certonly --nginx --key-type ecdsa \
--cert-name redbean-ecdsa -d redbean.dev -d www.redbean.dev
certbot certonly --nginx --key-type rsa \
--cert-name redbean-rsa -d redbean.dev -d www.redbean.dev
You can then program /var/www/html/.init.lua as such:
ProgramPrivateKey(Slurp('/etc/letsencrypt/live/redbean-ecdsa/privkey.pem')) ProgramCertificate(Slurp('/etc/letsencrypt/live/redbean-ecdsa/fullchain.pem')) ProgramPrivateKey(Slurp('/etc/letsencrypt/live/redbean-rsa/privkey.pem')) ProgramCertificate(Slurp('/etc/letsencrypt/live/redbean-rsa/fullchain.pem')) if IsDaemon() then ProgramUid(33) # see `vipw` to get appropriate number ProgramGid(33) # see `vigr` to get appropriate number ProgramPort(80) ProgramPort(443) ProgramLogPath('/var/log/redbean.log') ProgramPidPath('/var/run/redbean.pid') end function OnHttpRequest() path = GetPath() if path == '/favicon.ico' or path == '/site.webmanifest' or path == '/favicon-16x16.png' or path == '/favicon-32x32.png' or path == '/apple-touch-icon' then SetLogLevel(kLogWarn) end Route() SetHeader('Content-Language', 'en-US') end
You'd then run redbean as follows:
redbean.com -dD /var/www/html
You can load as many public and private keys as you want. They can be specified as pem, der, concatenated ascii, bundles, or chains. If you don't specify specific chains then redbean will automatically infer it based on SUBJECT → ISSUER relationships. Your redbean won't serve the self-signed root certificate at the end of the chain where self-signed is defined as SUBJECT == ISSUER. Otherwise you can control when chains terminate by setting the max length constraint to zero.
Your redbean supports SSL virtual hosting. 99.76% of TLS clients send a Server Name Indicator (SNI), which is matched against DNS or IPs in Subject Alternative Names (SAN) or the Common Name (CN) of subject if SAN isn't used. This means you don't need to reveal your whole domain portfolio to each client just to have ssl. You can just use different certificates for each domain if you choose to do so. If redbean can't find an appropriate match, then the first certificate will be chosen.
SSL layer client verification is unusual, but some options are:
-j to enable verification of HTTPS clients.
Clients are verified based on the SSL roots you've provided. Those
can be installed via the Lua API or placed in the ZIP executable
folder usr/share/ssl/root.
SSL verbosity is controlled as follows for troubleshooting:
-V log ssl errors -VV log ssl state changes too -VVV log ssl informational messages too -VVVV log ssl verbose details too
That's in addition to existing flags like -vvvm.
Once you have SSL setup, the threats you're most likely to face are distributed denial of service attacks, or DDOS for short. redbean defeats this form of abuse using token buckets and blackholed. That lets us protect not only your web app, but the kernel itself.
The example below shows how you can configure a secure reverse proxy. Each block of 256 IPs is allowed to send one message per second, on average, with bursts of up to ~100 messages per second allowed for a short period of time. Once an IP exhausts its tokens, it gets blocked by the system firewall.
-- we'll be reverse proxying to a server running here BACKEND = '127.0.0.1:8080' -- put your home or office public ip here -- so you don't accidentally ban yourself -- you don't want to lose your ssh access ProgramTrustedIp(ParseIp('1.2.3.4'), 32) RELAY_HEADERS_TO_CLIENT = { 'Access-Control-Allow-Origin', 'Cache-Control', 'Connection', 'Content-Type', 'Last-Modified', 'Referrer-Policy', } function OnServerStart() -- enables ddos protection ProgramTokenBucket() -- place limit on number of forked workers -- this determines the point at which "meltdown" happens assert(unix.setrlimit(unix.RLIMIT_NPROC, 1000, 1000)) end function OnWorkerStart() -- set limits on memory and cpu just in case assert(unix.setrlimit(unix.RLIMIT_RSS, 2*1024*1024)) assert(unix.setrlimit(unix.RLIMIT_CPU, 2)) -- we don't need filesystem access assert(unix.unveil(nil, nil)) -- we only need minimal system calls assert(unix.pledge("stdio inet unix", nil, unix.PLEDGE_PENALTY_RETURN_EPERM)) end function OnHttpRequest() local ip = GetClientAddr() if not IsTrustedIp(ip) then local tok = AcquireToken(ip) if tok < 2 then if Blackhole(ip) then Log(kLogWarn, "banned %s" % {FormatIp(ip)}) else Log(kLogWarn, "failed to ban %s" % {FormatIp(ip)}) end end if tok < 30 then ServeError(429) SetHeader('Connection', 'close') Log(kLogWarn, "warned %s who has %d tokens" % {FormatIp(ip), tok}) return end end local url = 'http://' .. BACKEND .. EscapePath(GetPath()) local name = GetParam('name') if name then url = url .. '?name=' .. EscapeParam(name) end local status, headers, body = Fetch(url, {method = GetMethod(), headers = { ['Accept'] = GetHeader('Accept'), ['CF-IPCountry'] = GetHeader('CF-IPCountry'), ['If-Modified-Since'] = GetHeader('If-Modified-Since'), ['Referer'] = GetHeader('Referer'), ['Sec-CH-UA-Platform'] = GetHeader('Sec-CH-UA-Platform'), ['User-Agent'] = GetHeader('User-Agent'), ['X-Forwarded-For'] = FormatIp(ip)}}) if status then SetStatus(status) for k,v in pairs(RELAY_HEADERS_TO_CLIENT) do SetHeader(v, headers[v]) end Write(body) else local err = headers Log(kLogError, "proxy failed %s" % {err}) ServeError(503) end end
You'll notice in the code above that we're doing token bucket logic by
hand. redbean actually does this automatically in the core code, but
it's only enforced on accept()'d connections. If you aren't specifying
SetHeader('Connection', 'close') on each response, then a
each of those connections can technically send infinite HTTP messages.
redbean core doesn't police messages, which is left to your Lua code
to decide.
The trick blackholed uses is the raw prerouting table,
which drops traffic before the kernel's conntrack mechanism
kicks in. This works out of the box with zero configuration on Linux
if you download the blackholed daemon from
the supplementary tools section above. Its
purpose is to permit unprivileged system accounts to safely ban IPv4
addresses via a UNIX named socket. redbean integrates with this
service automatically when ProgramTokenBucket() is used.
sudo ./blackholed.com -d
Please note you can't use Cloudflare if you use redbean's DDOS protection. If someone launches a DDOS attack against your server through Cloudflare, then redbean can't blackhole the IPs of the attackers because your server isn't talking to the attackers. We recommend using redbean's DDOS protection instead, because it's done a much better job fending off attacks than Cloudflare was able to do, for the services we operate online.
redbean provides hardened ASAN (Address Sanitizer) builds that proactively guard against any potential memory weaknesses that may be discovered, such as buffer overruns, use after free, etc. MODE=asan is recomended when serving on the public Internet.
redbean also supports robust sandboxing on Linux Kernel 5.13+ and OpenBSD. The recommended way to harden your redbean is to call the pledge() and unveil() functions.
Here's an example. If you have a SQLite app, then the key to using features like pledge() is to perform the permissions-demanding activities before calling pledge(). In this case, that would be connecting to the DB first.
function OnWorkerStart() db = sqlite3.open("db.sqlite3") db:busy_timeout(1000) db:exec[[PRAGMA journal_mode=WAL]] db:exec[[PRAGMA synchronous=NORMAL]] db:exec[[SELECT x FROM warmup WHERE x = 1]] assert(unix.setrlimit(unix.RLIMIT_RSS, 100 * 1024 * 1024)) assert(unix.setrlimit(unix.RLIMIT_CPU, 4)) assert(unix.unveil("/var/tmp", "rwc")) assert(unix.unveil("/tmp", "rwc")) assert(unix.unveil(nil, nil)) assert(unix.pledge("stdio flock rpath wpath cpath", nil, unix.PLEDGE_PENALTY_RETURN_EPERM)) end
What makes this technique interesting is redbean doesn't have file system access to the database file, and instead uses an inherited file descriptor that was opened beforehand. With SQLite the tmp access is only needed to support things like covering indexes. The -Z flag is also helpful to see where things go wrong, so you know which promises are needed to support your use case.
pledge() will work on all Linux kernels since RHEL6 since it uses SECCOMP BPF filtering. On the other hand, unveil() requires Landlock LSM which was only introduced in 2021. If you need unveil() then be sure to test the restrictions work. Most environments don't support unveil(), so it's designed to be a no-op in unsupported environments.
Alternatively, there's CLI flags which make it simple to get started:
unix.pledge("stdio rpath inet dns id") to
be called on workers after fork() is called. This permits
read-only operations and APIs like Fetch() that let workers send
and receive data with private and public Internet hosts. Access to
the unix module is somewhat restricted, disallowing its more
powerful APIs like exec.
unix.pledge("stdio rpath id") to be
called on workers after after fork() is called. This prevents
workers from talking to the network (other than the client) and
allows read-only file system access (e.g. -D DIR
flag). The `id` group helps you to call other functions important
to redbean security, such as the
unix.setrlimit() function.
unix.pledge("stdio") to be called on
workers after after fork() is called. This prevents workers from
communicating with the network (other than the client connection)
and prevents file system access (with some exceptions like
logging). Redbean should only be able to serve from its own zip
file in this mode. Lua script access to the unix module is highly
restricted.
Unlike the unix.pledge() function, these sandboxing flags use a more permissive policy on Linux. Rather than killing the process, they'll cause system calls to fail with EPERM instead. Therefore these flags should be gentler when you want security errors to be recoverable.
redbean contains software licensed ISC, MIT, BSD-2, BSD-3, zlib. The transitive closure of legal notices can be found inside the binary structure. We put the licenses inside the binary we believe that this satisfactorily automates legal compliance, for the redbean project and anyone who uses it.
# Note: Benchmarked on an AMD Ryzen Threadripper PRO 7995WX 96-Cores # Note: Use ./redbean.com -s $ wrk --latency -t 256 -c 256 -H 'Accept-Encoding: gzip' \ http://127.0.0.1:8080/favicon.ico Running 10s test @ http://127.0.0.1:8080/tool/net/demo/index.html 256 threads and 256 connections Thread Stats Avg Stdev Max +/- Stdev Latency 3.92ms 35.76ms 829.17ms 98.34% Req/Sec 25.72k 8.08k 39.35k 62.68% Latency Distribution 50% 29.00us 75% 42.00us 90% 63.00us 99% 159.62ms 35413605 requests in 6.60s, 210.36GB read Requests/sec: 5366223.78 Transfer/sec: 31.88GB
![[United States of Lemuria - two dollar bill - all debts public and primate]](http://worker.jart.workers.dev/sectorlisp2/lemuria.png)
Funding for redbean was crowdsourced from Justine Tunney's GitHub sponsors and Patreon subscribers. Your support is what makes projects like redbean possible. Thank you.