diff options
| author | Berke Güzel <wenekar1@gmail.com> | 2026-10-12 17:49:34 +0300 |
|---|---|---|
| committer | Berke Güzel <wenekar1@gmail.com> | 2026-10-12 17:49:34 +0300 |
| commit | 4384435941b8c7658856b381d071aee2e0035e89 (patch) | |
| tree | 5ece99c3dd1decb8ec3fd404c0ece2b5865a552a | |
term: move the term core to subprojects/term
The core now builds, tests, and benchmarks alone, as a Meson subproject
with no dependency on the main tree.
- include/term/ holds term.h and the header-only unicode/. src/ holds
the core and its private headers. The main tree gets only
include/term through term_dep, so it cannot reach a private header.
- The term tests, the term benchmarks, the Kitty parser suite, and
tools/amalgamate.py and tools/generate_unicode.py move with it. The
tests run as term:<name>.
- The main build passes its -march (arch_flag) and b_ndebug to the
subproject. A directory configured before this commit needs
meson configure -Dterm:arch_flag=-march=... once.
- The term tests compile with -UNDEBUG. They do work inside assert.
- The subproject exports term_test_dep with term_api.h, for the main
tree's render, thumbnail, color, and selection tests.
- The Kitty bridge used struct vector2i and has not compiled since the
public API redesign. It builds again. 2 of its 14 tests fail: REP
after a tab, and a cursor report after a resize.
- dirgraph.py drops the term and unicode layers. Its self-test uses pty.
- The symbols test ignores the __odr_asan symbols that ASan adds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
88 files changed, 30449 insertions, 0 deletions
diff --git a/README.md b/README.md new file mode 100644 index 0000000..e877a01 --- /dev/null +++ b/README.md @@ -0,0 +1,143 @@ +# term + +term is a headless terminal core: the parser, screen ring, logical-line +history store, write core, and viewport snapshots. It is a C11 library +with no dependency but libm. + +| Path | Contents | +| --- | --- | +| `include/term/` | The public headers: `term.h`, and the header-only `unicode/` | +| `src/` | The core. Its headers are private. | +| `tests/` | Unit, model, fuzz, and portability tests, and the Kitty parser suite | +| `benchmarks/` | `term-write-bench`, `term-resize-bench`, and `hist-bench` | +| `tools/` | `amalgamate.py` and `generate_unicode.py` | + +`meson setup build && meson test -C build` builds and tests it alone. + +## Embedding + +Pick one of these: + +- **Meson subproject**: `dependency('term')`, or + `subproject('term').get_variable('term_dep')`. The `arch_flag` option + sets the lowest instruction set, such as `-march=x86-64-v3`. +- **Amalgamation**: run `tools/amalgamate.py . OUT`. It writes + `OUT/term.c` and `OUT/term.h`. Compile `term.c` with any C11 compiler + and link libm. A Rust `build.rs` needs one + `cc::Build::new().file("term.c").compile("term")`. +- **Sources**: compile `src/*.c` and `src/osc/*.c` once, with `src` and + `include/term` on the include path. + +The library needs nothing else. The `*-amalgamation` tests run the term +tests against the generated `term.c`, and `term-cross-test` compiles it +for aarch64, macOS, and Windows. + +### SIMD level + +The hot paths have AVX2, AVX-512, and NEON versions. Without them, +scroll and line-feed output runs 8 to 30 times slower. The build needs +no flags to get them: + +- On x86-64 with GCC or Clang, each kernel compiles at every level with + a target attribute. The first `term_new()` picks the highest level the + CPU supports. The levels are `base` (x86-64), `avx2` (x86-64-v3), and + `avx512` (x86-64-v4 with VBMI and VBMI2). `term_isa()` names the level + in use. Set `TERM_HEADLESS_ISA` to a lower level to force it. +- `-march` sets the lowest level. The checks for that level and below + become constants, and their kernels inline into their callers. +- aarch64 always has NEON, so it needs no choice at run time. +- MSVC gets only the levels that its `/arch` flag enables. + +`simd.h` states the rules for a new kernel. + +- `term.h` is the only header a host includes. It includes + `term_input.h` and `term_types.h`. It compiles as C11 and as C++. +- Every global symbol starts with `term_`. The library builds with hidden + symbols, so a shared build exports only the `TERM_API` functions of + `term.h`. `term-symbols-test` checks both. +- Any thread can call any function. Calls on one terminal run one at a + time. `term.h` states the rules. +- The `static inline` helpers (`term_color_*`, `term_view_color`, + `term_attr_underline`) only decode documented bit layouts. A binding + generator such as bindgen skips them. Mirror them in the binding, or + use the generator's static-function wrappers. + +A host wires the terminal in four places: + +1. Create it with `term_new()`. Set `term_options.callbacks.reply` to + send replies to the program. +2. Feed program output with `term_feed()`. +3. Translate key and mouse events into `struct term_key_event` and + `struct term_mouse_event`. Encode them with `term_input_encode_key()` + and `term_input_encode_mouse()`, and send the bytes to the program. +4. Paint from a `struct term_snapshot`. The terminal gives code points, + colors, and attributes. The host shapes and draws the glyphs. + +## Files and responsibilities + +The files are in `src/`. + +| File | Responsibility | +| --- | --- | +| `term.c` | Mode queries, cursor options, mouse reports, focus events | +| `core.c`, `core.h` | Unified write core primitives (`sc_print_ascii`, `sc_linefeed`, etc.) | +| `screen.c`, `screen.h` | Screen ring buffer, line feed, editing, resize, and reflow | +| `hist.c`, `hist.h` | Logical-line history store, SIMD cell encoder, and page arena | +| `hist_view.c`, `hist_view.h` | Viewport anchor, history row lookup, and scroll offset | +| `snapshot.c` | Immutable view snapshot, row keys, and scroll movement detection | +| `selection.c` | Text selection, coordinate mapping, and UTF-8 export | +| `gstore.c`, `gstore.h` | Per-terminal grapheme cluster table | +| `text.c` | Byte stream decoder, control character dispatch, and text run routing | +| `text_run.c`, `text_run.h` | SIMD UTF-8 text run blit | +| `fastcsi.c` | Fast CSI sequence parser | +| `input.c` | Key and mouse encoding | +| `escape.c` | Parser state machine for ESC, CSI, DCS, and OSC | +| `pen.h` | SGR graphics rendition state | +| `osc/dispatch.c` | OSC command dispatcher and palette queries | +| `osc/link.c` | OSC 8 hyperlink table | +| `osc/clipboard.c` | OSC 52 clipboard reads and writes | +| `osc/misc.c` | Operating system commands: title, directory, notifications | + +## Architecture + +### Write core +All text and control sequences route through unified primitives in +`core.c`. Primitives handle character printing, line feeds, margins, +and cursor movement. + +### Screen and storage +The visible screen is a circular ring of rows (`struct screen`). +Rows that scroll off the top move into the logical-line history store +(`struct hist`). +The history store compacts lines into UTF-8 text and delta-encoded +attribute runs. + +### Viewport and snapshots +The viewport names a logical line and a wrapped segment, or the live bottom. +The anchor also keeps the cell offset before wrapping. +Output keeps the anchor on the same text. +Resize maps that cell offset to a segment at the new width. +Eviction clamps the anchor to the oldest retained line. +A line can lose its prefix when it exceeds the history byte limit. +The anchor tracks the removed cells to keep its text position. +The alternate screen has no history. + +Relative scroll walks from the anchor. +Absolute scroll uses a circular Fenwick index of line row counts. +Snapshot capture updates this index. +The write path does not update the index. +History descriptors keep row totals for the current width. +A width change starts wide lines with estimated row counts. +Snapshot capture makes the counts exact for lines in the view. +Counts above 254 rows use a separate cache. +That cache grows only when a visited line needs it. + +`term_snapshot_update` takes a mutable terminal. +It updates history query metadata before it copies the view. +Snapshot getters and history getters do not change terminal state. +Renderers do not read terminal memory directly. Renderers read an +immutable snapshot (`struct term_snapshot`). +Snapshots track row keys based on row content identities. +When output scrolls, snapshots detect the scroll distance `k`. +Renderers shift existing pixels and draw records by `k` rows and paint +only changed rows. diff --git a/benchmarks/hist_bench.c b/benchmarks/hist_bench.c new file mode 100644 index 0000000..2cf5608 --- /dev/null +++ b/benchmarks/hist_bench.c @@ -0,0 +1,93 @@ +#include "hist.h" +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <time.h> + +static double now(void) +{ + struct timespec ts; + clock_gettime(CLOCK_MONOTONIC, &ts); + return ts.tv_sec + ts.tv_nsec * 1e-9; +} + +/* Rows shaped like the term-write-bench lines that scroll into history. */ +static int make_row(const char *name, struct cell *row) +{ + struct cell d = { ' ', TERM_COLOR_FG, TERM_COLOR_BG, 0, 0 }; + int n = 0; + if (!strcmp(name, "ascii-row")) { + const char *s = "The quick brown fox jumps over the lazy dog. 0123456789"; + for (; s[n]; n++) { + row[n] = d; + row[n].c = (uint8_t)s[n]; + } + } else if (!strcmp(name, "cjk-row")) { + for (int i = 0; i < 23; i++) { + row[n] = d; + row[n].c = 0x65e5 + (uint32_t)i; + row[n++].flags = CELL_WIDE; + row[n] = d; + row[n].c = 0; + row[n++].flags = CELL_TRAIL; + } + } else if (!strcmp(name, "colors-row")) { + const char *s = "red RGB"; + for (; s[n]; n++) { + row[n] = d; + row[n].c = (uint8_t)s[n]; + if (n < 3) + row[n].fg = term_color_index(1); + if (n > 3) + row[n].fg = term_color_rgb(12, 34, 56); + } + } else if (!strcmp(name, "fg-char-row")) { + for (; n < 120; n++) { + row[n] = d; + row[n].c = 'a' + (uint32_t)(n % 26); + row[n].fg = term_color_rgb((uint8_t)n, (uint8_t)(n * 3), (uint8_t)(n * 7)); + } + } + return n; +} + +int main(int argc, char **argv) +{ + int iterations = argc > 1 ? atoi(argv[1]) : 10000000; + struct hist h; + if (!term_hist_init(&h, 10000, 8 * 1024 * 1024)) { + fprintf(stderr, "failed to initialize hist\n"); + return EXIT_FAILURE; + } + + const char *line = "abcdefghijklmnopqrstuvwxyz\n"; + int len = (int)strlen(line); + for (int i = 0; i < 10000; i++) + term_hist_push_ascii(&h, line, len, NULL, false); + double t0 = now(); + for (int i = 0; i < iterations; i++) + term_hist_push_ascii(&h, line, len, NULL, false); + double elapsed = now() - t0; + printf("%-12s %6.2f ns/line %.2f GiB/s (%d B/line)\n", "ascii", + elapsed * 1e9 / iterations, + (double)iterations * len / (1024.0 * 1024.0 * 1024.0) / elapsed, len); + term_hist_free(&h); + + static const char *rows[] = { "ascii-row", "cjk-row", "colors-row", "fg-char-row" }; + for (size_t k = 0; k < sizeof(rows) / sizeof(rows[0]); k++) { + struct cell row[128]; + int n = make_row(rows[k], row); + int iters = iterations / 4; + if (!term_hist_init(&h, 4096, 0)) + return EXIT_FAILURE; + for (int i = 0; i < 10000; i++) + term_hist_push_row(&h, row, n, false, NULL); + t0 = now(); + for (int i = 0; i < iters; i++) + term_hist_push_row(&h, row, n, false, NULL); + elapsed = now() - t0; + printf("%-12s %6.2f ns/row (%d cells)\n", rows[k], elapsed * 1e9 / iters, n); + term_hist_free(&h); + } + return EXIT_SUCCESS; +} diff --git a/benchmarks/meson.build b/benchmarks/meson.build new file mode 100644 index 0000000..ef78f28 --- /dev/null +++ b/benchmarks/meson.build @@ -0,0 +1,13 @@ +foreach entry : [ + ['term-write-bench', files('term_write_bench.c')], + ['term-resize-bench', files('term_resize_bench.c')], + ['hist-bench', files('hist_bench.c')], +] + benchmark(entry[0], + executable(entry[0], entry[1], + c_args: ['-D_POSIX_C_SOURCE=200809L'], + dependencies: term_internal_dep, + build_by_default: false), + timeout: 600, + ) +endforeach diff --git a/benchmarks/term_resize_bench.c b/benchmarks/term_resize_bench.c new file mode 100644 index 0000000..08634d1 --- /dev/null +++ b/benchmarks/term_resize_bench.c @@ -0,0 +1,202 @@ +#include "term/term.h" +#include <stdbool.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <time.h> + +#define BUDGET_MS_DEFAULT 0.5 +/* Rows written past the history limit, so the history is full and wraps. */ +#define FILL_EXTRA_ROWS 512 +#define MAX_SAMPLES 4096 + +/* + * History sizes to measure. 4096 is the size of the recorded baseline. + * The budget applies to every size. Today the larger sizes go over it; the + * terminal redesign targets a resize cost that does not grow with history. + */ +static const int history_sizes[] = { 100, 4096, 10000 }; + +static double now(void) +{ + struct timespec ts; + clock_gettime(CLOCK_MONOTONIC, &ts); + return ts.tv_sec + ts.tv_nsec * 1e-9; +} + +static int cmp_double(const void *a, const void *b) +{ + double x = *(const double *)a, y = *(const double *)b; + return x < y ? -1 : x > y; +} + +/* Content mixes, chosen so reflow cost spans no-op to worst case. */ +enum content { SHORT, WRAPPED, MIXED, PROMPT, WIDE, CONTENT_COUNT }; +static const char *content_names[] = {"short", "wrapped", "mixed", "prompt", "wide"}; + +static void emit_line(struct term *t, enum content kind, int n, int cols) +{ + char line[4096]; + int len = 0; + switch (kind) { + case SHORT: + len = snprintf(line, sizeof(line), "%d: short output line\r\n", n); + break; + case WRAPPED: + /* ~3.5 screen rows at 120 cols: every resize must rewrap these. */ + for (int i = 0; i < 420; i++) line[len++] = 'a' + (n + i) % 26; + line[len++] = '\r'; + line[len++] = '\n'; + break; + case MIXED: + if (n % 4 == 0) + for (int i = 0; i < 300; i++) line[len++] = 'a' + (n + i) % 26; + else + len = snprintf(line, sizeof(line), "%d: build step ok", n); + line[len++] = '\r'; + line[len++] = '\n'; + break; + case PROMPT: + /* OSC 133 marked prompt rows do not wrap; output rows do. */ + if (n % 3 == 0) + len = snprintf(line, sizeof(line), + "\033]133;A\033\\user@host:~/very/long/project/path $ \033]133;B\033\\" + "cargo build --release --features=all\r\n"); + else + len = snprintf(line, sizeof(line), " Compiling crate v0.%d.0\r\n", n % 100); + break; + case WIDE: + for (int i = 0; i < 60; i++) len += snprintf(line + len, sizeof(line) - len, "日本語"); + line[len++] = '\r'; + line[len++] = '\n'; + break; + default: + break; + } + (void)cols; + term_feed(t, line, (size_t)len, 0); +} + +static struct term *make_filled(enum content kind, int cols, int rows, int history) +{ + struct term *t = term_new(&(struct term_options){ + .size = { .x = cols, .y = rows }, + .scrollback = history, + }); + if (!t) return NULL; + for (int n = 0; n < history + FILL_EXTRA_ROWS; n++) + emit_line(t, kind, n, cols); + return t; +} + +static void resize(struct term *t, int cols, int rows) +{ + if (!term_resize(t, (struct term_vector2i){ .x = cols, .y = rows })) { + fprintf(stderr, "term_resize failed: out of memory\n"); + exit(EXIT_FAILURE); + } +} + +static void report(const char *content, const char *scenario, double *s, int n, double budget_ms) +{ + qsort(s, (size_t)n, sizeof(*s), cmp_double); + double p50 = s[n / 2]; + double p99 = s[(int)((double)(n - 1) * 0.99)]; + double max = s[n - 1]; + printf("%-8s %-10s p50 %7.3f p99 %7.3f max %7.3f ms %s\n", + content, scenario, p50, p99, max, max <= budget_ms ? "ok" : "OVER"); +} + +/* One resize per freshly filled terminal: content never decays across samples. */ +static double bench_pair(enum content kind, int from_cols, int to_cols, int rows, + int history, double *out, int samples) +{ + double worst = 0; + for (int i = 0; i < samples; i++) { + struct term *t = make_filled(kind, from_cols, rows, history); + if (!t) exit(EXIT_FAILURE); + double start = now(); + resize(t, to_cols, rows); + double ms = (now() - start) * 1e3; + term_free(t); + out[i] = ms; + if (ms > worst) worst = ms; + } + return worst; +} + +/* A drag mutates progressively, so every step is timed on the same terminal. */ +static int bench_drag(enum content kind, int from_cols, int to_cols, int rows, + int history, double *out, int passes) +{ + int n = 0; + int step = from_cols > to_cols ? -1 : 1; + for (int p = 0; p < passes; p++) { + struct term *t = make_filled(kind, from_cols, rows, history); + if (!t) exit(EXIT_FAILURE); + for (int c = from_cols + step; c != to_cols + step; c += step) { + double start = now(); + resize(t, c, rows); + out[n++] = (now() - start) * 1e3; + } + term_free(t); + } + return n; +} + +int main(int argc, char **argv) +{ + const char *only = argc > 1 && strcmp(argv[1], "all") ? argv[1] : NULL; + double budget = argc > 2 ? strtod(argv[2], NULL) : BUDGET_MS_DEFAULT; + int only_history = argc > 3 ? atoi(argv[3]) : 0; + if (argc > 4 || only_history < 0) { + fprintf(stderr, "usage: %s [content|all [budget-ms [history]]]\n", argv[0]); + return EXIT_FAILURE; + } + int rows = 40, samples = 32; + static const struct { const char *name; int from, to; } pairs[] = { + {"grow", 80, 120}, + {"shrink", 120, 80}, + {"double", 80, 160}, + {"halve", 160, 80}, + {"narrow", 120, 20}, + {"widen", 20, 120}, + }; + static double s[MAX_SAMPLES]; + bool matched = false, over = false; + + for (size_t h = 0; h < sizeof(history_sizes) / sizeof(history_sizes[0]); h++) { + int history = history_sizes[h]; + if (only_history && only_history != history) continue; + bool history_over = false; + printf("reflow budget %.3f ms, %d rows, scrollback %d\n\n", budget, rows, history); + for (int k = 0; k < CONTENT_COUNT; k++) { + if (only && strcmp(only, content_names[k])) continue; + matched = true; + for (size_t p = 0; p < sizeof(pairs) / sizeof(pairs[0]); p++) { + double worst = bench_pair((enum content)k, pairs[p].from, pairs[p].to, rows, + history, s, samples); + history_over |= worst > budget; + report(content_names[k], pairs[p].name, s, samples, budget); + } + int n = bench_drag((enum content)k, 120, 80, rows, history, s, 8); + for (int i = 0; i < n; i++) history_over |= s[i] > budget; + report(content_names[k], "drag", s, n, budget); + putchar('\n'); + } + printf("scrollback %d: %s\n\n", history, + history_over ? "OVER budget" : "within budget"); + over |= history_over; + } + if (!matched) { + if (only_history) + fprintf(stderr, "unknown content mix or history size: %s %d\n", + argv[1], only_history); + else + fprintf(stderr, "unknown content mix: %s\n", argv[1]); + return EXIT_FAILURE; + } + printf("%s\n", over ? "FAIL: at least one resize exceeded budget" + : "PASS: all resizes within budget"); + return over ? EXIT_FAILURE : EXIT_SUCCESS; +} diff --git a/benchmarks/term_write_bench.c b/benchmarks/term_write_bench.c new file mode 100644 index 0000000..50853f6 --- /dev/null +++ b/benchmarks/term_write_bench.c @@ -0,0 +1,184 @@ +#include "term/term.h" +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <time.h> + +static double now(void) +{ + struct timespec ts; + clock_gettime(CLOCK_MONOTONIC, &ts); + return ts.tv_sec + ts.tv_nsec * 1e-9; +} + +static int cmp_double(const void *a, const void *b) +{ + double d = *(const double *)a - *(const double *)b; + return (d > 0) - (d < 0); +} + +/* Timed sample: iterations full-buffer writes split at raw byte offsets, so a + * chunk smaller than len cuts UTF-8/CSI sequences mid-stream and exercises the + * parser's save-and-resume path, like real PTY reads do. */ +static double run_sample(struct term *t, const char *buf, size_t len, + size_t chunk, int iterations, unsigned flags) +{ + double start = now(); + for (int i = 0; i < iterations; i++) { + for (size_t pos = 0; pos < len; pos += chunk) + term_feed(t, buf + pos, len - pos < chunk ? len - pos : chunk, flags); + } + return now() - start; +} + +/* The history size of the recorded baseline. Keep it, so results stay + * comparable with the reports in benchmarks/. */ +enum { BENCH_SCROLLBACK = 4096 }; + +int main(int argc, char **argv) +{ + unsigned seed = argc > 2 ? (unsigned)strtoul(argv[2], NULL, 10) : 42; + size_t chunk = argc > 3 ? (size_t)strtoul(argv[3], NULL, 10) : 0; + int iterations = argc > 4 ? atoi(argv[4]) : 512; + int reps = argc > 5 ? atoi(argv[5]) : 7; + if (chunk > 65536 || iterations <= 0 || reps <= 0) { + fprintf(stderr, "usage: %s [case|all [seed [chunk (1..65536) [iterations (>0) [reps (>0)]]]]]\n" + "no chunk arg: each case runs whole-buffer and 4096-byte chunks\n", + argv[0]); + return EXIT_FAILURE; + } + static const struct { const char *name, *text; } cases[] = { + {"longline", NULL}, + {"manyline", NULL}, + {"lf-burst", NULL}, + {"crlf-burst", NULL}, + {"margin-lf", NULL}, + {"margin-crlf", NULL}, + {"ascii", "The quick brown fox jumps over the lazy dog. 0123456789\r\n"}, + {"utf8", "Türkçe: İstanbul — αβγ 日本語\r\n"}, + {"utf8var", "ab ér漢字 🙂 café x̂ 東京a\r\n"}, + {"cjk", "日本語漢字仮名交じり文中文測試終端輸出性能比較\r\n"}, + {"colors", "\033[31mred\033[0m \033[38;2;12;34;56mRGB\033[m\r\n"}, + {"fg-char", "\033[38;2;1;23;145ma\033[38;2;23;145;2mb\033[38;2;145;2;34mc"}, + {"fgbg-char", "\033[48;2;145;23;1m\033[38;2;1;23;145ma" + "\033[48;2;2;145;23m\033[38;2;23;145;2mb" + "\033[48;2;34;2;145m\033[38;2;145;2;34mc"}, + {"fg-rand", NULL}, + {"fgbg-rand", NULL}, + {"cursor", "\033[12;20Hhello\033[2DXY\033[K\033[1A\033[3C!\r\n"}, + {"erase", "\033[Hscreen\033[2J\033[Hnext\033[2K\r\n"}, + {"dch", "some text on a row\033[10D\033[4Ptail\r\n"}, + {"editor", NULL}, + {"random", NULL}, + }; + char buf[65536]; + double samples[64]; + if (reps > (int)(sizeof(samples) / sizeof(samples[0]))) + reps = sizeof(samples) / sizeof(samples[0]); + bool matched = false, header = false; + for (size_t c = 0; c < sizeof(cases) / sizeof(cases[0]); c++) { + if (argc > 1 && strcmp(argv[1], "all") && strcmp(argv[1], cases[c].name)) continue; + matched = true; + size_t len = sizeof(buf); + if (cases[c].text) { + size_t n = strlen(cases[c].text); + len = sizeof(buf) / n * n; + for (size_t i = 0; i < len; i += n) + memcpy(buf + i, cases[c].text, n); + } else if (!strncmp(cases[c].name, "fg", 2)) { + /* Random truecolor per character, like termbench's FGPerChar and + * FGBGPerChar. Repeating a few colors would let the branch + * predictor learn the digit-count branches. */ + bool bg = !strncmp(cases[c].name, "fgbg", 4); + unsigned state = seed; + size_t n = 0; + for (;;) { + char seq[64]; + int k = 0; + state = state * 1103515245u + 12345u; + unsigned r = (state >> 8) & 255, g = (state >> 16) & 255, b = (state >> 24) & 255; + if (bg) + k += snprintf(seq + k, sizeof(seq) - (size_t)k, "\033[48;2;%u;%u;%um", b, r, g); + k += snprintf(seq + k, sizeof(seq) - (size_t)k, "\033[38;2;%u;%u;%um%c", r, g, b, + 'a' + (int)(state % 26)); + if (n + (size_t)k > sizeof(buf)) break; + memcpy(buf + n, seq, (size_t)k); + n += (size_t)k; + } + len = n; + } else if (strstr(cases[c].name, "-burst") || + !strncmp(cases[c].name, "margin-", 7)) { + bool crlf = strstr(cases[c].name, "crlf") != NULL; + for (size_t i = 0; i < len; i++) + buf[i] = crlf && (i & 1) == 0 ? '\r' : '\n'; + } else if (!strcmp(cases[c].name, "editor")) { + /* Full-screen redraw: CUP to each row followed by a full line of + * text, like a vim/htop frame. */ + size_t n = 0; + for (int r = 1; r <= 40 && n + 140 < sizeof(buf); r++) { + n += (size_t)snprintf(buf + n, sizeof(buf) - n, "\033[%d;1H", r); + memset(buf + n, 'a' + r % 26, 110); + n += 110; + } + len = n; + } else { + unsigned state = seed; + unsigned choices = !strcmp(cases[c].name, "manyline") ? 27 : 26; + bool random = !strcmp(cases[c].name, "random"); + for (size_t i = 0; i < len; i++) { + state = state * 1103515245u + 12345u; + unsigned pick = (state >> 16) % choices; + buf[i] = random ? (char)(state >> 16) : pick == 26 ? '\n' : (char)('a' + pick); + } + } + if (!header) { + header = true; + /* MiB/s counts input bytes: bytes/cell and work/byte differ per + * case, so only compare rows sharing case and chunk size. */ + printf("# %-9s %6s %10s %10s %10s %8s\n", + "case", "chunk", "MiB/s med", "min", "max", "ns/B"); + } + size_t chunks[2] = {chunk, 4096}; + int nchunks = 1; + if (!chunk) { + chunks[0] = len; + nchunks = 2; + } + for (int k = 0; k < nchunks; k++) { + size_t ck = chunks[k]; + const char *env_cols = getenv("TERM_COLS"); + int cols_val = env_cols ? atoi(env_cols) : 120; + struct term *t = term_new(&(struct term_options){ + .size = { .x = cols_val, .y = 40 }, + .scrollback = BENCH_SCROLLBACK, + }); + if (!t) return EXIT_FAILURE; + /* manyline has bare LFs: each one also returns the cursor to + * column 0, as a tty with ONLCR does. */ + unsigned flags = !strcmp(cases[c].name, "manyline") ? TERM_FEED_ONLCR : 0; + if (!strncmp(cases[c].name, "margin-", 7)) { + const char setup[] = "\033[5;36r\033[36;1H"; + term_feed(t, setup, sizeof(setup) - 1, 0); + } else if (strstr(cases[c].name, "-burst")) { + const char setup[] = "\033[40;1H"; + term_feed(t, setup, sizeof(setup) - 1, 0); + } + /* Untimed warmup reaches steady-state scroll so rep 1 isn't a + * cold outlier. */ + run_sample(t, buf, len, ck, 16, flags); + for (int r = 0; r < reps; r++) + samples[r] = run_sample(t, buf, len, ck, iterations, flags); + qsort(samples, reps, sizeof(samples[0]), cmp_double); + double bytes = (double)len * iterations; + double med = bytes / (1024 * 1024) / samples[reps / 2]; + double min = bytes / (1024 * 1024) / samples[reps - 1]; + double max = bytes / (1024 * 1024) / samples[0]; + printf("%-9s %6zu %10.2f %10.2f %10.2f %8.3f\n", + cases[c].name, ck, med, min, max, + samples[reps / 2] * 1e9 / bytes); + term_free(t); + } + } + if (!matched) fprintf(stderr, "unknown benchmark case: %s\n", argv[1]); + return matched ? EXIT_SUCCESS : EXIT_FAILURE; +} diff --git a/include/term/term.h b/include/term/term.h new file mode 100644 index 0000000..4853ebf --- /dev/null +++ b/include/term/term.h @@ -0,0 +1,421 @@ +#ifndef TERM_H +#define TERM_H + +/* + * Terminal emulator core. + * + * A struct term parses the bytes that a program writes and keeps the screen, + * the history, and the modes. The application feeds bytes with term_feed() + * and reads state with the getters below. A renderer never reads the + * terminal directly: it copies the viewport into a struct term_snapshot and + * paints from that copy. + * + * All functions take the terminal as the first argument. Functions that take + * a const struct term do not change what it shows. A function that returns + * text copies it into a buffer that the caller owns. + * + * Threads. Any thread can call any function. Each call that reads or changes + * a terminal enters its writer section, so calls on one terminal run one at a + * time. A snapshot belongs to one thread at a time. term_free() is the last + * call on a terminal. + */ + +#include <stdbool.h> +#include <stddef.h> +#include <stdint.h> + +#include "term_input.h" +#include "term_types.h" + +#ifdef __cplusplus +extern "C" { +#endif + +struct term; +struct term_snapshot; + +/* A row holds at most this many cells. */ +#define TERM_MAX_COLS 65535 +/* The history limit when term_options.scrollback is 0. */ +#define TERM_SCROLLBACK_DEFAULT 10000 + +/* + * Colors. A term_color is either a direct color (alpha 0xFF, then RGB) or a + * palette slot: an index 0-255, the default foreground, or the default + * background. A slot resolves when a snapshot is painted, so a theme change + * recolors text that is already on screen. + */ +typedef uint32_t term_color; + +#define TERM_COLOR_SLOT 0x01000000u +/* The palette index of a slot; see struct term_view.palette. */ +#define TERM_COLOR_SLOT_INDEX 0x1FFu +#define TERM_COLOR_FG (TERM_COLOR_SLOT | 0x100u) +#define TERM_COLOR_BG (TERM_COLOR_SLOT | 0x101u) + +static inline term_color term_color_index(int n) +{ + return TERM_COLOR_SLOT | ((uint32_t)n & 0xFFu); +} + +static inline term_color term_color_rgb(uint8_t r, uint8_t g, uint8_t b) +{ + return 0xFF000000u | (uint32_t)r << 16 | (uint32_t)g << 8 | b; +} + +static inline bool term_color_is_slot(term_color c) +{ + return (c >> 24) == (TERM_COLOR_SLOT >> 24); +} + +/* Parse an X11 or OSC color specification ("#rgb", "rgb:r/g/b", ...). */ +TERM_API bool term_color_parse(const char *spec, uint32_t *argb); + +/* The colors a terminal starts with. All values are ARGB. An all-zero theme + * selects the built-in theme. */ +struct term_theme { + uint32_t fg, bg, cursor; + uint32_t ansi[16]; +}; + +/* Fill theme with the built-in theme. */ +TERM_API void term_theme_default(struct term_theme *theme); +/* Fill theme with the high-contrast theme. */ +TERM_API void term_theme_high_contrast(struct term_theme *theme); +/* Sets one palette key: foreground, background, or color0 to color15. + * An empty value resets the key to the built-in theme. False, with no + * change, for an unknown key or a bad color. */ +TERM_API bool term_theme_set_key(struct term_theme *p, const char *key, const char *val); + +enum term_cursor_style { + TERM_CURSOR_BLOCK, + TERM_CURSOR_UNDERLINE, + TERM_CURSOR_BAR, +}; + +/* Mouse reporting modes and encodings. The values are the DEC mode numbers. */ +enum term_mouse { + TERM_MOUSE_OFF = 0, + TERM_MOUSE_X10 = 9, + TERM_MOUSE_NORMAL = 1000, + TERM_MOUSE_BUTTON = 1002, + TERM_MOUSE_ANY = 1003, +}; + +enum term_mouse_encoding { + TERM_MOUSE_ENC_X10 = 0, + TERM_MOUSE_ENC_UTF8 = 1005, + TERM_MOUSE_ENC_SGR = 1006, + TERM_MOUSE_ENC_URXVT = 1015, + TERM_MOUSE_ENC_SGR_PIXELS = 1016, +}; + +/* + * Callbacks. They run inside term_feed() and term_focus(), on the thread + * that calls those functions. A callback must not call back into the + * terminal. String arguments are valid only during the call. + */ +struct term_callbacks { + void *ud; + /* Bytes the terminal sends back to the program (replies to queries). */ + void (*reply)(void *ud, const char *bytes, size_t len); + void (*title)(void *ud, const char *title); + /* OSC 52. selection names the target, such as "c". */ + void (*clipboard)(void *ud, const char *selection, const char *data, size_t len); + void (*notify)(void *ud, const char *title, const char *body); + void (*bell)(void *ud); + /* Parser trace lines for debugging. NULL turns tracing off. The line has no newline. */ + void (*trace)(void *ud, const char *line); +}; + +/* Options for term_new(). An all-zero value is valid. */ +struct term_options { + struct term_vector2i size; /* columns and rows; each is at least 1 */ + int scrollback; /* history lines; 0 selects TERM_SCROLLBACK_DEFAULT */ + struct term_theme theme; + enum term_cursor_style cursor_style; + bool cursor_steady; /* false: the cursor blinks */ + struct term_callbacks callbacks; +}; + +/* NULL when memory runs out. */ +/* The Unicode version of the width tables, such as "18.0.0". A host can + * pass it to programs as UNICODE_VERSION. */ +TERM_API const char *term_unicode_version(void); + +/* The length of the prefix of buf made of bytes in [0x20, hi). With hi + * 0x7f, it finds a run of printable ASCII in UTF-8 text, so a host can + * skip shaping for it. Any thread can call it. */ +TERM_API size_t term_scan_printable_span(const char *buf, size_t len, unsigned char hi); + +/* The instruction set level of the running code: "base", "avx2", "avx512", + * or "neon". On x86-64, the first term_new() picks the highest level the CPU + * supports. Set TERM_HEADLESS_ISA to a lower level to force it, for tests + * and comparisons. */ +TERM_API const char *term_isa(void); + +TERM_API struct term *term_new(const struct term_options *opt); +TERM_API void term_free(struct term *t); + +/* Hold the writer section across several calls, so that no other thread + * changes the terminal between them. The section is a spin lock: keep it + * short. The thread that holds it can call every function, and it can enter + * the section again. */ +TERM_API void term_lock(struct term *t); +TERM_API void term_unlock(struct term *t); + +/* Change the size in cells. Returns false when memory runs out; the terminal + * is then unchanged. The screen reflows to the new width. */ +TERM_API bool term_resize(struct term *t, struct term_vector2i size); + +/* Replace the theme. A different theme also drops the colors that the + * program set with OSC; the same theme changes nothing. */ +TERM_API void term_set_theme(struct term *t, const struct term_theme *theme); + +/* The cursor style that DECSCUSR 0 and a reset restore. */ +TERM_API void term_set_cursor_default(struct term *t, enum term_cursor_style style, bool steady); + +/* + * Bytes from the program. + * + * TERM_FEED_ONLCR: a line feed also returns the cursor to column 0, as a tty + * with OPOST and ONLCR does. Set it when the bytes did not pass through the + * tty output processing (for example the fast pipe). + */ +enum term_feed_flags { + TERM_FEED_ONLCR = 1u << 0, +}; + +TERM_API void term_feed(struct term *t, const char *bytes, size_t len, unsigned flags); + +/* Report a focus change. The terminal replies only in focus-event mode. */ +TERM_API void term_focus(struct term *t, bool focused); + +/* Modes that affect how the application encodes input. */ +struct term_modes { + unsigned keyboard; /* kitty keyboard protocol flags */ + enum term_mouse mouse; + enum term_mouse_encoding mouse_encoding; + struct term_vector2i grid; /* columns and rows */ + bool app_cursor; /* DECCKM */ + bool app_keypad; /* DECKPAM */ + bool backarrow_bs; /* DECBKM */ + bool bracketed_paste; + bool focus_events; + bool alternate; /* the alternate screen is active */ + bool reverse_video; /* DECSCNM */ + bool synchronized; /* mode 2026 */ +}; + +struct term_cursor { + struct term_vector2i pos; + enum term_cursor_style style; + bool visible; + bool blink; +}; + +TERM_API struct term_vector2i term_size(const struct term *t); +TERM_API struct term_modes term_modes(const struct term *t); + +/* + * Encode an input event as the bytes the program expects. Get modes from + * term_modes(). Record ev in keyboard with term_keyboard_update() first. Send + * the bytes to the program yourself. + */ +TERM_API struct term_input_result term_input_encode_key( + const struct term_keyboard_state *keyboard, + const struct term_key_event *ev, + const struct term_modes *modes, + char *buf, size_t capacity +); + +TERM_API struct term_input_result term_input_encode_mouse( + const struct term_mouse_event *ev, + const struct term_modes *modes, + char *buf, size_t capacity +); + +/* Bytes term_input_encode_paste() adds to the text at most. */ +#define TERM_PASTE_EXTRA 12 + +/* + * Encode pasted text. Drop control characters and bidi controls, so the + * program sees only what the user saw. Send a line break as CR. Add the + * bracketed-paste markers when the program asked for them. A buffer of + * len + TERM_PASTE_EXTRA bytes is always large enough. Text that is empty + * after the filter gives TERM_INPUT_CONSUMED. + */ +TERM_API struct term_input_result term_input_encode_paste( + const struct term_modes *modes, + const char *text, size_t len, + char *buf, size_t capacity +); +/* The cursor in screen coordinates (not the viewport). */ +TERM_API struct term_cursor term_cursor(const struct term *t); +/* Copy the directory from OSC 7 into buf, with the rules of + * term_selection_text(). Return 0 when there is none. */ +TERM_API size_t term_cwd(const struct term *t, char *buf, size_t cap); +/* Copy the pointer shape name from OSC 22 the same way. */ +TERM_API size_t term_pointer_shape(const struct term *t, char *buf, size_t cap); + +/* + * Viewport. The viewport shows as many rows as the screen. Offset 0 shows + * the live screen; a larger offset shows that many rows of history above it. + * The alternate screen has no history, so its offset is always 0. + */ +struct term_scrollback { + int offset; /* rows between the viewport and the live screen */ + int total; /* history rows at the current width */ +}; + +TERM_API struct term_scrollback term_scrollback(const struct term *t); +/* Scroll the viewport. A positive count moves back into history. */ +TERM_API void term_scroll(struct term *t, int lines); +TERM_API void term_scroll_to(struct term *t, int offset); + +/* + * Selection, in viewport coordinates. A selection starts at one cell and + * extends to another; the unit widens both ends to words or whole lines. + * The terminal clears the selection when the text under it changes. + */ +enum term_select_unit { + TERM_SELECT_CELL, + TERM_SELECT_WORD, + TERM_SELECT_LINE, +}; + +/* Start a selection at cell. For TERM_SELECT_CELL, right_half means the + * pointer is on the right half of the cell, so the selection starts after it. */ +TERM_API void term_select_start(struct term *t, struct term_vector2i cell, bool right_half, + enum term_select_unit unit); +/* Extend the selection from its start to cell. */ +TERM_API void term_select_extend(struct term *t, struct term_vector2i cell, bool right_half); +TERM_API void term_select_clear(struct term *t); +TERM_API bool term_has_selection(const struct term *t); +/* Copy the selected text as UTF-8 into buf. Return its length without the NUL. + * With a NULL buf or a zero cap, only the length is returned. Otherwise the + * copy is cut to cap - 1 bytes and NUL-terminated. Return 0 when nothing is + * selected. Hold term_lock across the length query and the copy to keep the + * length stable. */ +TERM_API size_t term_selection_text(const struct term *t, char *buf, size_t cap); +/* The OSC 8 link or the detected URL at a viewport cell. Copies it with the + * same rules as term_selection_text(). Return 0 when there is no link. */ +TERM_API size_t term_link_at(const struct term *t, struct term_vector2i cell, char *buf, size_t cap); + +/* Cell attributes. */ +enum term_attr { + TERM_ATTR_BOLD = 1u << 0, + TERM_ATTR_ITALIC = 1u << 1, + TERM_ATTR_FAINT = 1u << 2, + TERM_ATTR_STRIKE = 1u << 3, + TERM_ATTR_OVERLINE = 1u << 4, + TERM_ATTR_LINK = 1u << 5, + TERM_ATTR_EMOJI = 1u << 6, /* the text requested emoji presentation */ + TERM_ATTR_TEXT = 1u << 7, /* the text requested text presentation */ +}; + +#define TERM_ATTR_UNDERLINE_SHIFT 8 +#define TERM_ATTR_UNDERLINE_MASK (7u << TERM_ATTR_UNDERLINE_SHIFT) + +enum term_underline { + TERM_UNDERLINE_NONE, + TERM_UNDERLINE_SINGLE, + TERM_UNDERLINE_DOUBLE, + TERM_UNDERLINE_CURLY, + TERM_UNDERLINE_DOTTED, + TERM_UNDERLINE_DASHED, +}; + +static inline enum term_underline term_attr_underline(uint16_t attrs) +{ + return (enum term_underline)((attrs & TERM_ATTR_UNDERLINE_MASK) >> TERM_ATTR_UNDERLINE_SHIFT); +} + +/* text refers to a grapheme cluster in the snapshot; see term_snapshot_cluster(). */ +#define TERM_TEXT_CLUSTER 0x80000000u + +/* One cell as a snapshot shows it. */ +struct term_cell { + uint32_t text; /* a code point, TERM_TEXT_CLUSTER | n, or 0 for an empty cell */ + term_color fg, bg; + uint16_t attrs; /* enum term_attr and the underline style */ + uint8_t width; /* 1; 2 for a wide glyph; 0 for the right half of a wide glyph */ + uint8_t reserved; +}; + +#ifdef __cplusplus +static_assert(sizeof(struct term_cell) == 16, "four cells fill one 64-byte line"); +#else +_Static_assert(sizeof(struct term_cell) == 16, "four cells fill one 64-byte line"); +#endif + +/* + * Snapshot: a copy of the viewport for a renderer. It holds no pointer into + * the terminal, so it stays valid while the terminal changes. One terminal + * may have several snapshots, for example a window and a tab preview. + */ +TERM_API struct term_snapshot *term_snapshot_new(void); +TERM_API void term_snapshot_free(struct term_snapshot *s); + +/* Copy the viewport of t into s. Keep rows with unchanged content keys. + * Update the history view index and refine counts for visible wide lines. + * Return false when memory runs out. The snapshot is then empty. */ +TERM_API bool term_snapshot_update(struct term_snapshot *s, struct term *t); + +/* Palette entries of struct term_view.palette after the 256 indexed colors. */ +enum { + TERM_PALETTE_FG = 256, + TERM_PALETTE_BG = 257, + TERM_PALETTE_CURSOR = 258, + TERM_PALETTE_SIZE = 259, +}; + +struct term_view { + struct term_vector2i size; + struct term_cursor cursor; /* viewport coordinates; not visible when scrolled away */ + int scroll_offset; + /* Rows the content moved up since the previous update, when that movement + * explains the change; otherwise 0. */ + int scrolled; + bool reverse_video; + /* Changes when any part of the view changes. */ + uint64_t version; + /* Changes when a palette entry changes. */ + uint64_t palette_version; + const uint32_t *palette; /* TERM_PALETTE_SIZE ARGB entries */ +}; + +TERM_API const struct term_view *term_snapshot_view(const struct term_snapshot *s); +/* The cells of viewport row y: view->size.x entries. */ +TERM_API const struct term_cell *term_snapshot_row(const struct term_snapshot *s, int y); +/* An identity for the content of row y. Two rows with the same key hold the + * same cells, also when the content moved to another row. */ +TERM_API uint64_t term_snapshot_row_key(const struct term_snapshot *s, int y); + +/* Number of rows in history above the screen. 0 if alternate screen or no history. */ +TERM_API int term_history_rows(const struct term *t); +/* Copy up to max_rows history rows before the current viewport as UTF-8 into + * buf, with the same rules as term_selection_text(). Join soft-wrapped rows. + * End closed lines with a newline. This does not change the selection. + * Return 0 when there is no history text, and also when the text cannot be + * built. */ +TERM_API size_t term_history_text(struct term *t, int max_rows, char *buf, size_t cap); +TERM_API struct term_span term_snapshot_row_selection(const struct term_snapshot *s, int y); + +/* The code points of a cluster cell. Returns the number of code points and + * writes at most cap of them. */ +TERM_API int term_snapshot_cluster(const struct term_snapshot *s, uint32_t text, uint32_t *cps, int cap); + +/* The ARGB value of a color in a view. */ +static inline uint32_t term_view_color(const struct term_view *v, term_color c) +{ + if (!term_color_is_slot(c)) + return c; + return v->palette[c & TERM_COLOR_SLOT_INDEX]; +} + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/include/term/term_input.h b/include/term/term_input.h new file mode 100644 index 0000000..4b0a8d9 --- /dev/null +++ b/include/term/term_input.h @@ -0,0 +1,196 @@ +#ifndef TERM_INPUT_TYPES_H +#define TERM_INPUT_TYPES_H + +#include <stdbool.h> +#include <stddef.h> +#include <stdint.h> +#include "term_types.h" + +#ifdef __cplusplus +extern "C" { +#endif + +enum term_key { + TERM_KEY_NONE = 0, + + TERM_KEY_RETURN, + TERM_KEY_BACKSPACE, + TERM_KEY_TAB, + TERM_KEY_ESCAPE, + + TERM_KEY_UP, + TERM_KEY_DOWN, + TERM_KEY_RIGHT, + TERM_KEY_LEFT, + TERM_KEY_HOME, + TERM_KEY_END, + TERM_KEY_PAGE_UP, + TERM_KEY_PAGE_DOWN, + TERM_KEY_INSERT, + TERM_KEY_DELETE, + + TERM_KEY_F1, TERM_KEY_F2, TERM_KEY_F3, TERM_KEY_F4, TERM_KEY_F5, + TERM_KEY_F6, TERM_KEY_F7, TERM_KEY_F8, TERM_KEY_F9, TERM_KEY_F10, + TERM_KEY_F11, TERM_KEY_F12, TERM_KEY_F13, TERM_KEY_F14, TERM_KEY_F15, + TERM_KEY_F16, TERM_KEY_F17, TERM_KEY_F18, TERM_KEY_F19, TERM_KEY_F20, + TERM_KEY_F21, TERM_KEY_F22, TERM_KEY_F23, TERM_KEY_F24, TERM_KEY_F25, + TERM_KEY_F26, TERM_KEY_F27, TERM_KEY_F28, TERM_KEY_F29, TERM_KEY_F30, + TERM_KEY_F31, TERM_KEY_F32, TERM_KEY_F33, TERM_KEY_F34, TERM_KEY_F35, + + TERM_KEY_KP_0, TERM_KEY_KP_1, TERM_KEY_KP_2, TERM_KEY_KP_3, TERM_KEY_KP_4, + TERM_KEY_KP_5, TERM_KEY_KP_6, TERM_KEY_KP_7, TERM_KEY_KP_8, TERM_KEY_KP_9, + TERM_KEY_KP_DECIMAL, + TERM_KEY_KP_ENTER, + TERM_KEY_KP_ADD, + TERM_KEY_KP_SUBTRACT, + TERM_KEY_KP_MULTIPLY, + TERM_KEY_KP_DIVIDE, + TERM_KEY_KP_EQUAL, + TERM_KEY_KP_SEPARATOR, + TERM_KEY_KP_LEFT, + TERM_KEY_KP_RIGHT, + TERM_KEY_KP_UP, + TERM_KEY_KP_DOWN, + TERM_KEY_KP_PAGE_UP, + TERM_KEY_KP_PAGE_DOWN, + TERM_KEY_KP_HOME, + TERM_KEY_KP_END, + TERM_KEY_KP_INSERT, + TERM_KEY_KP_DELETE, + TERM_KEY_KP_BEGIN, + + TERM_KEY_CAPS_LOCK, + TERM_KEY_SCROLL_LOCK, + TERM_KEY_NUM_LOCK, + TERM_KEY_PRINT_SCREEN, + TERM_KEY_PAUSE, + TERM_KEY_MENU, + + TERM_KEY_MEDIA_PLAY, + TERM_KEY_MEDIA_PAUSE, + TERM_KEY_MEDIA_STOP, + TERM_KEY_MEDIA_FORWARD, + TERM_KEY_MEDIA_REWIND, + TERM_KEY_MEDIA_NEXT, + TERM_KEY_MEDIA_PREV, + TERM_KEY_MEDIA_RECORD, + TERM_KEY_VOLUME_DOWN, + TERM_KEY_VOLUME_UP, + TERM_KEY_MUTE, + + TERM_KEY_LSHIFT, + TERM_KEY_LCTRL, + TERM_KEY_LALT, + TERM_KEY_LSUPER, + TERM_KEY_LHYPER, + TERM_KEY_LMETA, + TERM_KEY_RSHIFT, + TERM_KEY_RCTRL, + TERM_KEY_RALT, + TERM_KEY_RSUPER, + TERM_KEY_RHYPER, + TERM_KEY_RMETA, + TERM_KEY_ISO_LEVEL3_SHIFT, + TERM_KEY_ISO_LEVEL5_SHIFT, + + TERM_KEY_COUNT +}; + +enum term_mod { + TERM_MOD_SHIFT = 1, + TERM_MOD_ALT = 2, + TERM_MOD_CTRL = 4, + TERM_MOD_SUPER = 8, + TERM_MOD_HYPER = 16, + TERM_MOD_META = 32, + TERM_MOD_CAPS = 64, + TERM_MOD_NUM = 128, +}; + +enum term_key_action { + TERM_KEY_PRESS = 1, + TERM_KEY_REPEAT = 2, + TERM_KEY_RELEASE = 3, +}; + +enum term_mouse_button { + TERM_MOUSE_BUTTON_PRIMARY = 0, + TERM_MOUSE_BUTTON_MIDDLE, + TERM_MOUSE_BUTTON_SECONDARY, + TERM_MOUSE_BUTTON_NONE, + TERM_MOUSE_BUTTON_EXTRA_1, + TERM_MOUSE_BUTTON_EXTRA_2, + TERM_MOUSE_BUTTON_EXTRA_3, + TERM_MOUSE_BUTTON_EXTRA_4, + TERM_MOUSE_BUTTON_EXTRA_5, + TERM_MOUSE_WHEEL_UP = 16, + TERM_MOUSE_WHEEL_DOWN, + TERM_MOUSE_WHEEL_LEFT, + TERM_MOUSE_WHEEL_RIGHT, +}; + +enum term_mouse_event_kind { + TERM_MOUSE_PRESS = 0, + TERM_MOUSE_RELEASE, + TERM_MOUSE_MOTION, + TERM_MOUSE_SCROLL, + TERM_MOUSE_LEAVE, +}; + +struct term_key_event { + enum term_key_action action; + uint16_t key; /* enum term_key; TERM_KEY_NONE for a text key */ + uint16_t keypad; /* the TERM_KEY_KP_* key when the key is on the keypad */ + unsigned mods; /* enum term_mod */ + uint32_t codepoint; /* the code point of the key without modifiers */ + uint32_t shifted; /* the code point of the key with Shift */ + uint32_t base; /* the code point of the key in the US layout */ + uint32_t text; /* the code point the key types, or 0 */ + bool text_deferred; /* an input method sends the text later */ +}; + +struct term_mouse_event { + struct term_vector2i pos; /* Grid cell coordinates (col, row) */ + struct term_vector2i pixel; /* Absolute pixel position in terminal grid */ + int button; + unsigned mods; + int scroll_lines; + float scroll_delta; + enum term_mouse_event_kind kind; +}; + +/* The keys that are down. An all-zero value has no keys down. */ +struct term_keyboard_state { + uint64_t down[(TERM_KEY_COUNT + 63) / 64]; +}; + +/* Record a press, repeat, or release in s. */ +TERM_API void term_keyboard_update(struct term_keyboard_state *s, const struct term_key_event *ev); +TERM_API bool term_keyboard_key_down(const struct term_keyboard_state *s, enum term_key key); + +/* Kitty keyboard protocol flags, as in term_modes.keyboard. */ +enum term_keyboard_flag { + TERM_KEYBOARD_DISAMBIGUATE = 1, + TERM_KEYBOARD_EVENT_TYPES = 2, + TERM_KEYBOARD_ALTERNATE_KEYS = 4, + TERM_KEYBOARD_ALL_KEYS = 8, + TERM_KEYBOARD_ASSOCIATED_TEXT = 16, +}; + +enum term_input_status { + TERM_INPUT_UNHANDLED = 0, /* the terminal does not use the event */ + TERM_INPUT_CONSUMED, /* the terminal uses the event but sends nothing */ + TERM_INPUT_BYTES, /* send the len bytes in buf to the program */ + TERM_INPUT_BUFFER_FULL, /* buf is too small; len is 0 */ +}; + +struct term_input_result { + size_t len; + enum term_input_status status; +}; + +#ifdef __cplusplus +} +#endif + +#endif /* TERM_INPUT_TYPES_H */ diff --git a/include/term/term_types.h b/include/term/term_types.h new file mode 100644 index 0000000..78be371 --- /dev/null +++ b/include/term/term_types.h @@ -0,0 +1,32 @@ +#ifndef TERM_TYPES_H +#define TERM_TYPES_H + +/* Marks the public functions. The library builds with hidden symbols, so + * only these are visible outside a shared build. A host can define it + * first, for example as __declspec(dllimport). */ +#ifndef TERM_API +#if defined(__GNUC__) || defined(__clang__) +#define TERM_API __attribute__((visibility("default"))) +#else +#define TERM_API +#endif +#endif + +#ifdef __cplusplus +extern "C" { +#endif + +struct term_vector2i { + int x, y; +}; + +struct term_span { + int begin; + int end; +}; + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/include/term/unicode/grapheme.h b/include/term/unicode/grapheme.h new file mode 100644 index 0000000..5573949 --- /dev/null +++ b/include/term/unicode/grapheme.h @@ -0,0 +1,26 @@ +#ifndef TERM_UNICODE_GRAPHEME_H +#define TERM_UNICODE_GRAPHEME_H + +#include <stdbool.h> +#include <stddef.h> +#include <stdint.h> + +#define GRAPHEME_MAX_LEN 20 +#define GRAPHEME_HANDLE_BASE 0x7F000000u + +static inline bool grapheme_is_handle(uint32_t c) +{ + return c >= GRAPHEME_HANDLE_BASE; +} + +static inline bool grapheme_regional_indicator(uint32_t cp) +{ + return cp >= 0x1F1E6 && cp <= 0x1F1FF; +} + +static inline bool grapheme_emoji_modifier(uint32_t cp) +{ + return cp >= 0x1F3FB && cp <= 0x1F3FF; +} + +#endif diff --git a/include/term/unicode/unicode.h b/include/term/unicode/unicode.h new file mode 100644 index 0000000..932cbf6 --- /dev/null +++ b/include/term/unicode/unicode.h @@ -0,0 +1,6 @@ +#ifndef TERM_UNICODE_UNICODE_H +#define TERM_UNICODE_UNICODE_H + +#include "width.h" + +#endif diff --git a/include/term/unicode/unicode_data.inc b/include/term/unicode/unicode_data.inc new file mode 100644 index 0000000..8c3d394 --- /dev/null +++ b/include/term/unicode/unicode_data.inc @@ -0,0 +1,258 @@ +/* Generated from Unicode 18.0.0 data by tools/generate_unicode.py. */ + +#define UNICODE_DATA_VERSION "18.0.0" + +static const struct unicode_range zero_width[] = { + {0xAD,0xAD},{0x300,0x36F},{0x483,0x489},{0x591,0x5BD}, + {0x5BF,0x5BF},{0x5C1,0x5C2},{0x5C4,0x5C5},{0x5C7,0x5C9}, + {0x600,0x605},{0x610,0x61A},{0x61C,0x61C},{0x64B,0x65F}, + {0x670,0x670},{0x6D6,0x6DD},{0x6DF,0x6E4},{0x6E7,0x6E8}, + {0x6EA,0x6ED},{0x70F,0x70F},{0x711,0x711},{0x730,0x74A}, + {0x7A6,0x7B0},{0x7EB,0x7F3},{0x7FD,0x7FD},{0x816,0x819}, + {0x81B,0x823},{0x825,0x827},{0x829,0x82D},{0x859,0x85B}, + {0x890,0x891},{0x897,0x89F},{0x8CA,0x902},{0x93A,0x93A}, + {0x93C,0x93C},{0x941,0x948},{0x94D,0x94D},{0x951,0x957}, + {0x962,0x963},{0x981,0x981},{0x9BC,0x9BC},{0x9C1,0x9C4}, + {0x9CD,0x9CD},{0x9E2,0x9E3},{0x9FE,0x9FE},{0xA01,0xA02}, + {0xA3C,0xA3C},{0xA41,0xA42},{0xA47,0xA48},{0xA4B,0xA4D}, + {0xA51,0xA51},{0xA70,0xA71},{0xA75,0xA75},{0xA81,0xA82}, + {0xABC,0xABC},{0xAC1,0xAC5},{0xAC7,0xAC8},{0xACD,0xACD}, + {0xAE2,0xAE3},{0xAFA,0xAFF},{0xB01,0xB01},{0xB3C,0xB3C}, + {0xB3F,0xB3F},{0xB41,0xB44},{0xB4D,0xB4D},{0xB53,0xB56}, + {0xB62,0xB63},{0xB82,0xB82},{0xBC0,0xBC0},{0xBCD,0xBCD}, + {0xC00,0xC00},{0xC04,0xC04},{0xC3C,0xC3C},{0xC3E,0xC40}, + {0xC46,0xC48},{0xC4A,0xC4D},{0xC55,0xC56},{0xC62,0xC63}, + {0xC81,0xC81},{0xCBC,0xCBC},{0xCBF,0xCBF},{0xCC6,0xCC6}, + {0xCCC,0xCCD},{0xCE2,0xCE3},{0xD00,0xD01},{0xD3B,0xD3C}, + {0xD41,0xD44},{0xD4D,0xD4D},{0xD62,0xD63},{0xD81,0xD81}, + {0xDCA,0xDCA},{0xDD2,0xDD4},{0xDD6,0xDD6},{0xE31,0xE31}, + {0xE34,0xE3A},{0xE47,0xE4E},{0xEB1,0xEB1},{0xEB4,0xEBC}, + {0xEC8,0xECE},{0xF18,0xF19},{0xF35,0xF35},{0xF37,0xF37}, + {0xF39,0xF39},{0xF71,0xF7E},{0xF80,0xF84},{0xF86,0xF87}, + {0xF8D,0xF97},{0xF99,0xFBC},{0xFC6,0xFC6},{0x102D,0x1030}, + {0x1032,0x1037},{0x1039,0x103A},{0x103D,0x103E},{0x1058,0x1059}, + {0x105E,0x1060},{0x1071,0x1074},{0x1082,0x1082},{0x1085,0x1086}, + {0x108D,0x108D},{0x109D,0x109D},{0x1160,0x11FF},{0x135D,0x135F}, + {0x1712,0x1714},{0x1732,0x1733},{0x1752,0x1753},{0x1772,0x1773}, + {0x17B4,0x17B5},{0x17B7,0x17BD},{0x17C6,0x17C6},{0x17C9,0x17D3}, + {0x17DD,0x17DD},{0x180B,0x180F},{0x1885,0x1886},{0x18A9,0x18A9}, + {0x1920,0x1922},{0x1927,0x1928},{0x1932,0x1932},{0x1939,0x193B}, + {0x1A17,0x1A18},{0x1A1B,0x1A1B},{0x1A56,0x1A56},{0x1A58,0x1A5E}, + {0x1A60,0x1A60},{0x1A62,0x1A62},{0x1A65,0x1A6C},{0x1A73,0x1A7C}, + {0x1A7F,0x1A7F},{0x1AB0,0x1AF0},{0x1B00,0x1B03},{0x1B34,0x1B34}, + {0x1B36,0x1B3A},{0x1B3C,0x1B3C},{0x1B42,0x1B42},{0x1B6B,0x1B73}, + {0x1B80,0x1B81},{0x1BA2,0x1BA5},{0x1BA8,0x1BA9},{0x1BAB,0x1BAD}, + {0x1BE6,0x1BE6},{0x1BE8,0x1BE9},{0x1BED,0x1BED},{0x1BEF,0x1BF1}, + {0x1C2C,0x1C33},{0x1C36,0x1C37},{0x1CD0,0x1CD2},{0x1CD4,0x1CE0}, + {0x1CE2,0x1CE8},{0x1CED,0x1CED},{0x1CF4,0x1CF4},{0x1CF8,0x1CF9}, + {0x1DC0,0x1DFF},{0x200B,0x200F},{0x202A,0x202E},{0x2060,0x2064}, + {0x2066,0x206F},{0x20D0,0x20F0},{0x2CEF,0x2CF1},{0x2D7F,0x2D7F}, + {0x2DE0,0x2DFF},{0x302A,0x302D},{0x3099,0x309A},{0xA66F,0xA672}, + {0xA674,0xA67D},{0xA69E,0xA69F},{0xA6F0,0xA6F1},{0xA802,0xA802}, + {0xA806,0xA806},{0xA80B,0xA80B},{0xA825,0xA826},{0xA82C,0xA82C}, + {0xA8C4,0xA8C5},{0xA8E0,0xA8F1},{0xA8FF,0xA8FF},{0xA926,0xA92D}, + {0xA947,0xA951},{0xA980,0xA982},{0xA9B3,0xA9B3},{0xA9B6,0xA9B9}, + {0xA9BC,0xA9BD},{0xA9E5,0xA9E5},{0xAA29,0xAA2E},{0xAA31,0xAA32}, + {0xAA35,0xAA36},{0xAA43,0xAA43},{0xAA4C,0xAA4C},{0xAA7C,0xAA7C}, + {0xAAB0,0xAAB0},{0xAAB2,0xAAB4},{0xAAB7,0xAAB8},{0xAABE,0xAABF}, + {0xAAC1,0xAAC1},{0xAAEC,0xAAED},{0xAAF6,0xAAF6},{0xABE5,0xABE5}, + {0xABE8,0xABE8},{0xABED,0xABED},{0xD7B0,0xD7C6},{0xD7CB,0xD7FB}, + {0xFB1E,0xFB1E},{0xFE00,0xFE0F},{0xFE20,0xFE2F},{0xFEFF,0xFEFF}, + {0xFFF9,0xFFFB},{0x101FD,0x101FD},{0x102E0,0x102E0},{0x10376,0x1037A}, + {0x10A01,0x10A03},{0x10A05,0x10A06},{0x10A0C,0x10A0F},{0x10A38,0x10A3A}, + {0x10A3F,0x10A3F},{0x10AE5,0x10AE6},{0x10D24,0x10D27},{0x10D69,0x10D6D}, + {0x10EAB,0x10EAC},{0x10ECB,0x10ECF},{0x10EF0,0x10EFF},{0x10F46,0x10F50}, + {0x10F82,0x10F85},{0x11001,0x11001},{0x11038,0x11046},{0x11070,0x11070}, + {0x11073,0x11074},{0x1107F,0x11081},{0x110B3,0x110B6},{0x110B9,0x110BA}, + {0x110BD,0x110BD},{0x110C2,0x110C2},{0x110CD,0x110CD},{0x11100,0x11102}, + {0x11127,0x1112B},{0x1112D,0x11134},{0x11173,0x11173},{0x11180,0x11181}, + {0x111B6,0x111BE},{0x111C9,0x111CC},{0x111CF,0x111CF},{0x1122F,0x11231}, + {0x11234,0x11234},{0x11236,0x11237},{0x1123E,0x1123E},{0x11241,0x11241}, + {0x112DF,0x112DF},{0x112E3,0x112EA},{0x11300,0x11301},{0x1133B,0x1133C}, + {0x11340,0x11340},{0x11366,0x1136C},{0x11370,0x11374},{0x113BB,0x113C0}, + {0x113CE,0x113CE},{0x113D0,0x113D0},{0x113D2,0x113D2},{0x113E1,0x113E2}, + {0x11438,0x1143F},{0x11442,0x11444},{0x11446,0x11446},{0x1145E,0x1145E}, + {0x114B3,0x114B8},{0x114BA,0x114BA},{0x114BF,0x114C0},{0x114C2,0x114C3}, + {0x115B2,0x115B5},{0x115BC,0x115BD},{0x115BF,0x115C0},{0x115DC,0x115DD}, + {0x11633,0x1163A},{0x1163D,0x1163D},{0x1163F,0x11640},{0x116AB,0x116AB}, + {0x116AD,0x116AD},{0x116B0,0x116B5},{0x116B7,0x116B7},{0x1171D,0x1171D}, + {0x1171F,0x1171F},{0x11722,0x11725},{0x11727,0x1172B},{0x1182F,0x11837}, + {0x11839,0x1183A},{0x1193B,0x1193C},{0x1193E,0x1193E},{0x11943,0x11943}, + {0x119D4,0x119D7},{0x119DA,0x119DB},{0x119E0,0x119E0},{0x11A01,0x11A0A}, + {0x11A33,0x11A38},{0x11A3B,0x11A3E},{0x11A47,0x11A47},{0x11A51,0x11A56}, + {0x11A59,0x11A5B},{0x11A8A,0x11A96},{0x11A98,0x11A99},{0x11B60,0x11B60}, + {0x11B62,0x11B64},{0x11B66,0x11B66},{0x11C30,0x11C36},{0x11C38,0x11C3D}, + {0x11C3F,0x11C3F},{0x11C92,0x11CA7},{0x11CAA,0x11CB0},{0x11CB2,0x11CB3}, + {0x11CB5,0x11CB6},{0x11D31,0x11D36},{0x11D3A,0x11D3A},{0x11D3C,0x11D3D}, + {0x11D3F,0x11D45},{0x11D47,0x11D47},{0x11D90,0x11D91},{0x11D95,0x11D95}, + {0x11D97,0x11D97},{0x11DF0,0x11DF0},{0x11EF3,0x11EF4},{0x11F00,0x11F01}, + {0x11F36,0x11F3A},{0x11F40,0x11F40},{0x11F42,0x11F42},{0x11F5A,0x11F5A}, + {0x13430,0x13440},{0x13447,0x13455},{0x1611E,0x16129},{0x1612D,0x1612F}, + {0x16AF0,0x16AF4},{0x16B30,0x16B36},{0x16F4F,0x16F4F},{0x16F8F,0x16F92}, + {0x16FE4,0x16FE4},{0x1BC9D,0x1BC9E},{0x1BCA0,0x1BCA3},{0x1CF00,0x1CF2D}, + {0x1CF30,0x1CF46},{0x1D127,0x1D128},{0x1D167,0x1D169},{0x1D173,0x1D182}, + {0x1D185,0x1D18B},{0x1D1AA,0x1D1AD},{0x1D242,0x1D244},{0x1D25B,0x1D25C}, + {0x1DA00,0x1DA36},{0x1DA3B,0x1DA6C},{0x1DA75,0x1DA75},{0x1DA84,0x1DA84}, + {0x1DA9B,0x1DA9F},{0x1DAA1,0x1DAAF},{0x1E000,0x1E006},{0x1E008,0x1E018}, + {0x1E01B,0x1E021},{0x1E023,0x1E024},{0x1E026,0x1E02A},{0x1E08F,0x1E08F}, + {0x1E130,0x1E136},{0x1E2AE,0x1E2AE},{0x1E2EC,0x1E2EF},{0x1E4EC,0x1E4EF}, + {0x1E5EE,0x1E5EF},{0x1E6E3,0x1E6E3},{0x1E6E6,0x1E6E6},{0x1E6EE,0x1E6EF}, + {0x1E6F5,0x1E6F5},{0x1E8D0,0x1E8D6},{0x1E944,0x1E94A},{0xE0001,0xE0001}, + {0xE0020,0xE007F},{0xE0100,0xE01EF}, +}; + +static const struct unicode_range nonprintable[] = { + {0x0,0x1F},{0x7F,0x9F},{0xD800,0xDFFF}, +}; + +static const struct unicode_range wide[] = { + {0x1100,0x115F},{0x231A,0x231B},{0x2329,0x232A},{0x23E9,0x23EC}, + {0x23F0,0x23F0},{0x23F3,0x23F3},{0x25FD,0x25FE},{0x2614,0x2615}, + {0x2630,0x2637},{0x2648,0x2653},{0x267F,0x267F},{0x268A,0x268F}, + {0x2693,0x2693},{0x26A1,0x26A1},{0x26AA,0x26AB},{0x26BD,0x26BE}, + {0x26C4,0x26C5},{0x26CE,0x26CE},{0x26D4,0x26D4},{0x26EA,0x26EA}, + {0x26F2,0x26F3},{0x26F5,0x26F5},{0x26FA,0x26FA},{0x26FD,0x26FD}, + {0x2705,0x2705},{0x270A,0x270B},{0x2728,0x2728},{0x274C,0x274C}, + {0x274E,0x274E},{0x2753,0x2755},{0x2757,0x2757},{0x2795,0x2797}, + {0x27B0,0x27B0},{0x27BF,0x27BF},{0x2B1B,0x2B1C},{0x2B50,0x2B50}, + {0x2B55,0x2B55},{0x2E80,0x2E99},{0x2E9B,0x2EF3},{0x2F00,0x2FD5}, + {0x2FF0,0x303E},{0x3041,0x3096},{0x3099,0x30FF},{0x3105,0x312F}, + {0x3131,0x318E},{0x3190,0x31E5},{0x31EF,0x321E},{0x3220,0x3247}, + {0x3250,0xA48C},{0xA490,0xA4C6},{0xA960,0xA97C},{0xAC00,0xD7A3}, + {0xF900,0xFAFF},{0xFE10,0xFE19},{0xFE30,0xFE52},{0xFE54,0xFE66}, + {0xFE68,0xFE6B},{0xFF01,0xFF60},{0xFFE0,0xFFE6},{0x16FE0,0x16FE4}, + {0x16FF0,0x16FF6},{0x17000,0x18CDA},{0x18CFF,0x18D20},{0x18D80,0x18DF2}, + {0x18E00,0x19191},{0x191A0,0x191D2},{0x1AFF0,0x1AFF3},{0x1AFF5,0x1AFFB}, + {0x1AFFD,0x1AFFE},{0x1B000,0x1B128},{0x1B132,0x1B132},{0x1B150,0x1B152}, + {0x1B155,0x1B155},{0x1B164,0x1B168},{0x1B170,0x1B2FB},{0x1D300,0x1D356}, + {0x1D360,0x1D376},{0x1F004,0x1F004},{0x1F0CF,0x1F0CF},{0x1F18E,0x1F18E}, + {0x1F191,0x1F19A},{0x1F1AE,0x1F1AE},{0x1F200,0x1F202},{0x1F210,0x1F23B}, + {0x1F240,0x1F248},{0x1F250,0x1F251},{0x1F260,0x1F265},{0x1F300,0x1F320}, + {0x1F32D,0x1F335},{0x1F337,0x1F37C},{0x1F37E,0x1F393},{0x1F3A0,0x1F3CA}, + {0x1F3CF,0x1F3D3},{0x1F3E0,0x1F3F0},{0x1F3F4,0x1F3F4},{0x1F3F8,0x1F43E}, + {0x1F440,0x1F440},{0x1F442,0x1F4FC},{0x1F4FF,0x1F53D},{0x1F54B,0x1F54E}, + {0x1F550,0x1F567},{0x1F57A,0x1F57A},{0x1F595,0x1F596},{0x1F5A4,0x1F5A4}, + {0x1F5FB,0x1F64F},{0x1F680,0x1F6C5},{0x1F6CC,0x1F6CC},{0x1F6D0,0x1F6D2}, + {0x1F6D5,0x1F6D9},{0x1F6DC,0x1F6DF},{0x1F6EB,0x1F6EC},{0x1F6F4,0x1F6FC}, + {0x1F7DA,0x1F7DA},{0x1F7E0,0x1F7EB},{0x1F7F0,0x1F7F0},{0x1F90C,0x1F93A}, + {0x1F93C,0x1F945},{0x1F947,0x1F9FF},{0x1FA70,0x1FA7C},{0x1FA80,0x1FAC6}, + {0x1FAC8,0x1FAC8},{0x1FACC,0x1FADD},{0x1FADF,0x1FAEB},{0x1FAEF,0x1FAFA}, + {0x20000,0x2FFFD},{0x30000,0x3FFFD}, +}; + +static const struct unicode_range spacing_mark[] = { + {0x903,0x903},{0x93B,0x93B},{0x93E,0x940},{0x949,0x94C}, + {0x94E,0x94F},{0x982,0x983},{0x9BE,0x9C0},{0x9C7,0x9C8}, + {0x9CB,0x9CC},{0x9D7,0x9D7},{0xA03,0xA03},{0xA3E,0xA40}, + {0xA83,0xA83},{0xABE,0xAC0},{0xAC9,0xAC9},{0xACB,0xACC}, + {0xB02,0xB03},{0xB3E,0xB3E},{0xB40,0xB40},{0xB47,0xB48}, + {0xB4B,0xB4C},{0xB57,0xB57},{0xBBE,0xBBF},{0xBC1,0xBC2}, + {0xBC6,0xBC8},{0xBCA,0xBCC},{0xBD7,0xBD7},{0xC01,0xC03}, + {0xC41,0xC44},{0xC82,0xC83},{0xCBE,0xCBE},{0xCC0,0xCC4}, + {0xCC7,0xCC8},{0xCCA,0xCCB},{0xCD5,0xCD6},{0xCF3,0xCF3}, + {0xD02,0xD03},{0xD3E,0xD40},{0xD46,0xD48},{0xD4A,0xD4C}, + {0xD57,0xD57},{0xD82,0xD83},{0xDCF,0xDD1},{0xDD8,0xDDF}, + {0xDF2,0xDF3},{0xF3E,0xF3F},{0xF7F,0xF7F},{0x102B,0x102C}, + {0x1031,0x1031},{0x1038,0x1038},{0x103B,0x103C},{0x1056,0x1057}, + {0x1062,0x1064},{0x1067,0x106D},{0x1083,0x1084},{0x1087,0x108C}, + {0x108F,0x108F},{0x109A,0x109C},{0x1715,0x1715},{0x1734,0x1734}, + {0x17B6,0x17B6},{0x17BE,0x17C5},{0x17C7,0x17C8},{0x1923,0x1926}, + {0x1929,0x192B},{0x1930,0x1931},{0x1933,0x1938},{0x1A19,0x1A1A}, + {0x1A55,0x1A55},{0x1A57,0x1A57},{0x1A61,0x1A61},{0x1A63,0x1A64}, + {0x1A6D,0x1A72},{0x1B04,0x1B04},{0x1B35,0x1B35},{0x1B3B,0x1B3B}, + {0x1B3D,0x1B41},{0x1B43,0x1B44},{0x1B82,0x1B82},{0x1BA1,0x1BA1}, + {0x1BA6,0x1BA7},{0x1BAA,0x1BAA},{0x1BE7,0x1BE7},{0x1BEA,0x1BEC}, + {0x1BEE,0x1BEE},{0x1BF2,0x1BF3},{0x1C24,0x1C2B},{0x1C34,0x1C35}, + {0x1CE1,0x1CE1},{0x1CF7,0x1CF7},{0x302E,0x302F},{0xA823,0xA824}, + {0xA827,0xA827},{0xA880,0xA881},{0xA8B4,0xA8C3},{0xA952,0xA953}, + {0xA983,0xA983},{0xA9B4,0xA9B5},{0xA9BA,0xA9BB},{0xA9BE,0xA9C0}, + {0xAA2F,0xAA30},{0xAA33,0xAA34},{0xAA4D,0xAA4D},{0xAA7B,0xAA7B}, + {0xAA7D,0xAA7D},{0xAAEB,0xAAEB},{0xAAEE,0xAAEF},{0xAAF5,0xAAF5}, + {0xABE3,0xABE4},{0xABE6,0xABE7},{0xABE9,0xABEA},{0xABEC,0xABEC}, + {0x11000,0x11000},{0x11002,0x11002},{0x11082,0x11082},{0x110B0,0x110B2}, + {0x110B7,0x110B8},{0x1112C,0x1112C},{0x11145,0x11146},{0x11182,0x11182}, + {0x111B3,0x111B5},{0x111BF,0x111C0},{0x111CE,0x111CE},{0x1122C,0x1122E}, + {0x11232,0x11233},{0x11235,0x11235},{0x112E0,0x112E2},{0x11302,0x11303}, + {0x1133E,0x1133F},{0x11341,0x11344},{0x11347,0x11348},{0x1134B,0x1134D}, + {0x11357,0x11357},{0x11362,0x11363},{0x113B8,0x113BA},{0x113C2,0x113C2}, + {0x113C5,0x113C5},{0x113C7,0x113CA},{0x113CC,0x113CD},{0x113CF,0x113CF}, + {0x11435,0x11437},{0x11440,0x11441},{0x11445,0x11445},{0x114B0,0x114B2}, + {0x114B9,0x114B9},{0x114BB,0x114BE},{0x114C1,0x114C1},{0x115AF,0x115B1}, + {0x115B8,0x115BB},{0x115BE,0x115BE},{0x11630,0x11632},{0x1163B,0x1163C}, + {0x1163E,0x1163E},{0x116AC,0x116AC},{0x116AE,0x116AF},{0x116B6,0x116B6}, + {0x1171E,0x1171E},{0x11720,0x11721},{0x11726,0x11726},{0x1182C,0x1182E}, + {0x11838,0x11838},{0x11930,0x11935},{0x11937,0x11938},{0x1193D,0x1193D}, + {0x11940,0x11940},{0x11942,0x11942},{0x119D1,0x119D3},{0x119DC,0x119DF}, + {0x119E4,0x119E4},{0x11A39,0x11A39},{0x11A57,0x11A58},{0x11A97,0x11A97}, + {0x11B61,0x11B61},{0x11B65,0x11B65},{0x11B67,0x11B67},{0x11C2F,0x11C2F}, + {0x11C3E,0x11C3E},{0x11CA9,0x11CA9},{0x11CB1,0x11CB1},{0x11CB4,0x11CB4}, + {0x11D8A,0x11D8E},{0x11D93,0x11D94},{0x11D96,0x11D96},{0x11EF5,0x11EF6}, + {0x11F03,0x11F03},{0x11F34,0x11F35},{0x11F3E,0x11F3F},{0x11F41,0x11F41}, + {0x1612A,0x1612C},{0x16F51,0x16F87},{0x16FF0,0x16FF1},{0x1D165,0x1D166}, + {0x1D16D,0x1D172},{0x1D250,0x1D252},{0x1D25F,0x1D25F},{0x1D280,0x1D281}, +}; + +static const struct unicode_range extended_pictographic[] = { + {0xA9,0xA9},{0xAE,0xAE},{0x203C,0x203C},{0x2049,0x2049}, + {0x2122,0x2122},{0x2139,0x2139},{0x2194,0x2199},{0x21A9,0x21AA}, + {0x231A,0x231B},{0x2328,0x2328},{0x23CF,0x23CF},{0x23E9,0x23F3}, + {0x23F8,0x23FA},{0x24C2,0x24C2},{0x25AA,0x25AB},{0x25B6,0x25B6}, + {0x25C0,0x25C0},{0x25FB,0x25FE},{0x2600,0x2604},{0x260E,0x260E}, + {0x2611,0x2611},{0x2614,0x2615},{0x2618,0x2618},{0x261D,0x261D}, + {0x2620,0x2620},{0x2622,0x2623},{0x2626,0x2626},{0x262A,0x262A}, + {0x262E,0x262F},{0x2638,0x263A},{0x2640,0x2640},{0x2642,0x2642}, + {0x2648,0x2653},{0x265F,0x2660},{0x2663,0x2663},{0x2665,0x2666}, + {0x2668,0x2668},{0x267B,0x267B},{0x267E,0x267F},{0x2692,0x2697}, + {0x2699,0x2699},{0x269B,0x269C},{0x26A0,0x26A1},{0x26A7,0x26A7}, + {0x26AA,0x26AB},{0x26B0,0x26B1},{0x26BD,0x26BE},{0x26C4,0x26C5}, + {0x26C8,0x26C8},{0x26CE,0x26CF},{0x26D1,0x26D1},{0x26D3,0x26D4}, + {0x26E9,0x26EA},{0x26F0,0x26F5},{0x26F7,0x26FA},{0x26FD,0x26FD}, + {0x2702,0x2702},{0x2705,0x2705},{0x2708,0x270D},{0x270F,0x270F}, + {0x2712,0x2712},{0x2714,0x2714},{0x2716,0x2716},{0x271D,0x271D}, + {0x2721,0x2721},{0x2728,0x2728},{0x2733,0x2734},{0x2744,0x2744}, + {0x2747,0x2747},{0x274C,0x274C},{0x274E,0x274E},{0x2753,0x2755}, + {0x2757,0x2757},{0x2763,0x2764},{0x2795,0x2797},{0x27A1,0x27A1}, + {0x27B0,0x27B0},{0x27BF,0x27BF},{0x2934,0x2935},{0x2B05,0x2B07}, + {0x2B1B,0x2B1C},{0x2B50,0x2B50},{0x2B55,0x2B55},{0x3030,0x3030}, + {0x303D,0x303D},{0x3297,0x3297},{0x3299,0x3299},{0x1F004,0x1F004}, + {0x1F02C,0x1F02F},{0x1F094,0x1F09F},{0x1F0AF,0x1F0B0},{0x1F0C0,0x1F0C0}, + {0x1F0CF,0x1F0D0},{0x1F0F6,0x1F0FF},{0x1F170,0x1F171},{0x1F17E,0x1F17F}, + {0x1F18E,0x1F18E},{0x1F191,0x1F19A},{0x1F1AF,0x1F1E5},{0x1F201,0x1F20F}, + {0x1F21A,0x1F21A},{0x1F22F,0x1F22F},{0x1F232,0x1F23A},{0x1F23C,0x1F23F}, + {0x1F249,0x1F25F},{0x1F266,0x1F321},{0x1F324,0x1F393},{0x1F396,0x1F397}, + {0x1F399,0x1F39B},{0x1F39E,0x1F3F0},{0x1F3F3,0x1F3F5},{0x1F3F7,0x1F3FA}, + {0x1F400,0x1F4FD},{0x1F4FF,0x1F53D},{0x1F549,0x1F54E},{0x1F550,0x1F567}, + {0x1F56F,0x1F570},{0x1F573,0x1F57A},{0x1F587,0x1F587},{0x1F58A,0x1F58D}, + {0x1F590,0x1F590},{0x1F595,0x1F596},{0x1F5A4,0x1F5A5},{0x1F5A8,0x1F5A8}, + {0x1F5B1,0x1F5B2},{0x1F5BC,0x1F5BC},{0x1F5C2,0x1F5C4},{0x1F5D1,0x1F5D3}, + {0x1F5DC,0x1F5DE},{0x1F5E1,0x1F5E1},{0x1F5E3,0x1F5E3},{0x1F5E8,0x1F5E8}, + {0x1F5EF,0x1F5EF},{0x1F5F3,0x1F5F3},{0x1F5FA,0x1F64F},{0x1F680,0x1F6C5}, + {0x1F6CB,0x1F6D2},{0x1F6D5,0x1F6E5},{0x1F6E9,0x1F6E9},{0x1F6EB,0x1F6F0}, + {0x1F6F3,0x1F6FF},{0x1F7DC,0x1F7F0},{0x1F80C,0x1F80F},{0x1F848,0x1F84F}, + {0x1F85A,0x1F85F},{0x1F888,0x1F88F},{0x1F8AE,0x1F8AF},{0x1F8BC,0x1F8BF}, + {0x1F8C2,0x1F8CF},{0x1F8D9,0x1F8FF},{0x1F90C,0x1F93A},{0x1F93C,0x1F945}, + {0x1F947,0x1F9FF},{0x1FA58,0x1FA5F},{0x1FA6E,0x1FAFF},{0x1FC00,0x1FFFD}, +}; + +static const struct unicode_range emoji_presentation[] = { + {0x231A,0x231B},{0x23E9,0x23EC},{0x23F0,0x23F0},{0x23F3,0x23F3}, + {0x25FD,0x25FE},{0x2614,0x2615},{0x2648,0x2653},{0x267F,0x267F}, + {0x2693,0x2693},{0x26A1,0x26A1},{0x26AA,0x26AB},{0x26BD,0x26BE}, + {0x26C4,0x26C5},{0x26CE,0x26CE},{0x26D4,0x26D4},{0x26EA,0x26EA}, + {0x26F2,0x26F3},{0x26F5,0x26F5},{0x26FA,0x26FA},{0x26FD,0x26FD}, + {0x2705,0x2705},{0x270A,0x270B},{0x2728,0x2728},{0x274C,0x274C}, + {0x274E,0x274E},{0x2753,0x2755},{0x2757,0x2757},{0x2795,0x2797}, + {0x27B0,0x27B0},{0x27BF,0x27BF},{0x2B1B,0x2B1C},{0x2B50,0x2B50}, + {0x2B55,0x2B55},{0x1F004,0x1F004},{0x1F0CF,0x1F0CF},{0x1F18E,0x1F18E}, + {0x1F191,0x1F19A},{0x1F1E6,0x1F1FF},{0x1F201,0x1F201},{0x1F21A,0x1F21A}, + {0x1F22F,0x1F22F},{0x1F232,0x1F236},{0x1F238,0x1F23A},{0x1F250,0x1F251}, + {0x1F300,0x1F320},{0x1F32D,0x1F335},{0x1F337,0x1F37C},{0x1F37E,0x1F393}, + {0x1F3A0,0x1F3CA},{0x1F3CF,0x1F3D3},{0x1F3E0,0x1F3F0},{0x1F3F4,0x1F3F4}, + {0x1F3F8,0x1F43E},{0x1F440,0x1F440},{0x1F442,0x1F4FC},{0x1F4FF,0x1F53D}, + {0x1F54B,0x1F54E},{0x1F550,0x1F567},{0x1F57A,0x1F57A},{0x1F595,0x1F596}, + {0x1F5A4,0x1F5A4},{0x1F5FB,0x1F64F},{0x1F680,0x1F6C5},{0x1F6CC,0x1F6CC}, + {0x1F6D0,0x1F6D2},{0x1F6D5,0x1F6D9},{0x1F6DC,0x1F6DF},{0x1F6EB,0x1F6EC}, + {0x1F6F4,0x1F6FC},{0x1F7E0,0x1F7EB},{0x1F7F0,0x1F7F0},{0x1F90C,0x1F93A}, + {0x1F93C,0x1F945},{0x1F947,0x1F9FF},{0x1FA70,0x1FA7C},{0x1FA80,0x1FAC6}, + {0x1FAC8,0x1FAC8},{0x1FACC,0x1FADD},{0x1FADF,0x1FAEB},{0x1FAEF,0x1FAFA}, +}; diff --git a/include/term/unicode/utf8.h b/include/term/unicode/utf8.h new file mode 100644 index 0000000..1c3cc3f --- /dev/null +++ b/include/term/unicode/utf8.h @@ -0,0 +1,243 @@ +#ifndef UTF_8_H +#define UTF_8_H + +#include <stdint.h> +#include <stddef.h> +#include <stdlib.h> +#include <string.h> + +#define UTF8_ACCEPT 0 +#define UTF8_REJECT 1 + +#define UTF8_3BYTE_MIN 0x800u +#define UTF8_SURROGATE_LO 0xd800u +#define UTF8_SURROGATE_HI 0xdfffu + +static const uint8_t utf8d[] = { + 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, + 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, + 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, + 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, + 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9, + 7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7, + 8,8,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2, + 0xa,0x3,0x3,0x3,0x3,0x3,0x3,0x3,0x3,0x3,0x3,0x3,0x3,0x4,0x3,0x3, + 0xb,0x6,0x6,0x6,0x5,0x8,0x8,0x8,0x8,0x8,0x8,0x8,0x8,0x8,0x8,0x8, + 0x0,0x1,0x2,0x3,0x5,0x8,0x7,0x1,0x1,0x1,0x4,0x6,0x1,0x1,0x1,0x1, + 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,1,1,1,1,1,0,1,0,1,1,1,1,1,1, + 1,2,1,1,1,1,1,2,1,2,1,1,1,1,1,1,1,1,1,1,1,1,1,2,1,1,1,1,1,1,1,1, + 1,2,1,1,1,1,1,1,1,2,1,1,1,1,1,1,1,1,1,1,1,1,1,3,1,3,1,1,1,1,1,1, + 1,3,1,1,1,1,1,3,1,3,1,1,1,1,1,1,1,3,1,1,1,1,1,1,1,1,1,1,1,1,1,1, +}; + +static inline uint32_t utf8_decode(uint32_t *state, uint32_t *codep, uint32_t byte) { + byte &= 0xff; + uint32_t type = utf8d[byte]; + + *codep = (*state != UTF8_ACCEPT) ? + (byte & 0x3fu) | (*codep << 6) : + (0xff >> type) & (byte); + + *state = utf8d[256 + *state*16 + type]; + return *state; +} + +static const uint8_t utf8_acc_state[256] = { +0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, +0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, +0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, +0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, +0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, +0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, +0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, +0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, +1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, +1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, +1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, +1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, +1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, +2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, +4, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 5, 3, 3, +6, 7, 7, 7, 8, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, +}; +static const uint8_t utf8_acc_codep[256] = { +0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, +16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, +32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, +48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, +64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, +80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, +96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, +112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, +0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, +0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, +0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, +0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, +0, 0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, +16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, +0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, +0, 1, 2, 3, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, +}; + +/* + * Decode one code point at the start of buf and return the bytes it used. + * Each maximal invalid subpart gives U+FFFD (Unicode 3.9); the byte that + * ends a subpart stays unused, so the caller reads it again. A C0 control + * ends an unfinished sequence the same way. An unfinished sequence at the + * end of buf gives *cp = 0 and keeps the state. + */ +static inline int utf8_decode_ground(uint32_t *state, uint32_t *codep, + const char *buf, size_t len, uint32_t *cp) { + uint32_t st = *state; + uint32_t cpv = *codep; + int n = 0; + *cp = 0; + if (len == 0) + return 0; + do { + unsigned char c = (unsigned char)buf[n]; + if (c < 0x20) { + if (st != UTF8_ACCEPT) *cp = 0xFFFD; + st = UTF8_ACCEPT; + break; + } + uint32_t before = st; + n++; + if (st == UTF8_ACCEPT) { + cpv = utf8_acc_codep[c]; + st = utf8_acc_state[c]; + } else { + uint32_t type = utf8d[c]; + cpv = (c & 0x3fu) | (cpv << 6); + st = utf8d[256 + st*16 + type]; + } + if (st == UTF8_ACCEPT) { *cp = cpv; break; } + if (st == UTF8_REJECT) { + st = UTF8_ACCEPT; + *cp = 0xFFFD; + if (before != UTF8_ACCEPT) n--; + break; + } + } while ((size_t)n < len); + *state = st; + *codep = cpv; + return n; +} + +static inline int +utf8_decode_one(const char *buf, size_t len, uint32_t *codep) +{ + const unsigned char *s = (const unsigned char *)buf; + unsigned c0, c1, c2, c3; + uint32_t cp; + + if (len == 0) + return 0; + c0 = s[0]; + if (c0 < 0x80) { + *codep = c0; + return 1; + } + if (c0 < 0xc2 || c0 > 0xf4) + return 0; + + if (c0 <= 0xdf) { + if (len < 2) + return 0; + c1 = s[1]; + if ((c1 & 0xc0) != 0x80) + return 0; + *codep = ((c0 & 0x1f) << 6) | (c1 & 0x3f); + return 2; + } + + if (c0 <= 0xef) { + if (len < 3) + return 0; + c1 = s[1]; + c2 = s[2]; + if ((c1 & 0xc0) != 0x80 || (c2 & 0xc0) != 0x80) + return 0; + cp = ((c0 & 0x0f) << 12) | ((c1 & 0x3f) << 6) | (c2 & 0x3f); + if (cp < UTF8_3BYTE_MIN || + (cp >= UTF8_SURROGATE_LO && cp <= UTF8_SURROGATE_HI)) + return 0; + *codep = cp; + return 3; + } + + if (len < 4) + return 0; + c1 = s[1]; + c2 = s[2]; + c3 = s[3]; + if ((c1 & 0xc0) != 0x80 || (c2 & 0xc0) != 0x80 || + (c3 & 0xc0) != 0x80) + return 0; + cp = ((c0 & 0x07) << 18) | ((c1 & 0x3f) << 12) | + ((c2 & 0x3f) << 6) | (c3 & 0x3f); + if (cp < 0x10000 || cp > 0x10ffff) + return 0; + *codep = cp; + return 4; +} + +static inline void +utf8_sanitize(char *dst, size_t cap, const char *src) +{ + if (!cap) + return; + size_t i = 0, len = strlen(src), off = 0; + while (off < len && i + 1 < cap) { + uint32_t cp; + int n = utf8_decode_one(src + off, len - off, &cp); + if (n <= 0 || i + (size_t)n >= cap) { + off++; + continue; + } + memcpy(dst + i, src + off, (size_t)n); + i += (size_t)n; + off += (size_t)n; + } + dst[i] = '\0'; +} + +/* A sanitized copy of src. Sanitizing only drops bytes, so the copy is + * never longer than src. */ +static inline char *utf8_sanitize_dup(const char *src) +{ + size_t cap = strlen(src) + 1; + char *dst = malloc(cap); + if (dst) + utf8_sanitize(dst, cap, src); + return dst; +} + +#define UTF8_MAX_BYTES 4 +static inline int utf8_encode(uint32_t cp, char *out) +{ + if (cp > 0x10ffff || (cp >= 0xd800 && cp <= 0xdfff)) + cp = 0xfffd; + if (cp < 0x80) { + out[0] = (char)cp; + return 1; + } + if (cp < 0x800) { + out[0] = (char)(0xc0 | (cp >> 6)); + out[1] = (char)(0x80 | (cp & 0x3f)); + return 2; + } + if (cp < 0x10000) { + out[0] = (char)(0xe0 | (cp >> 12)); + out[1] = (char)(0x80 | ((cp >> 6) & 0x3f)); + out[2] = (char)(0x80 | (cp & 0x3f)); + return 3; + } + out[0] = (char)(0xf0 | (cp >> 18)); + out[1] = (char)(0x80 | ((cp >> 12) & 0x3f)); + out[2] = (char)(0x80 | ((cp >> 6) & 0x3f)); + out[3] = (char)(0x80 | (cp & 0x3f)); + return 4; +} + +#endif diff --git a/include/term/unicode/width.h b/include/term/unicode/width.h new file mode 100644 index 0000000..fc639c8 --- /dev/null +++ b/include/term/unicode/width.h @@ -0,0 +1,55 @@ +#ifndef TERM_UNICODE_WIDTH_H +#define TERM_UNICODE_WIDTH_H + +#include <stdbool.h> +#include <stddef.h> +#include <stdint.h> + +struct unicode_range { uint32_t lo, hi; }; + +#include "unicode_data.inc" + +static inline bool in_ranges(const struct unicode_range *ranges, size_t count, uint32_t cp) +{ + size_t lo = 0, hi = count; + while (lo < hi) { + size_t mid = lo + (hi - lo) / 2; + if (cp < ranges[mid].lo) + hi = mid; + else if (cp > ranges[mid].hi) + lo = mid + 1; + else + return true; + } + return false; +} + +#define IN_TABLE(table, cp) in_ranges(table, sizeof(table) / sizeof(table[0]), cp) + +static inline int unicode_width(uint32_t cp) +{ + if (cp > 0x10FFFF || IN_TABLE(nonprintable, cp)) + return -1; + if (IN_TABLE(zero_width, cp)) + return 0; + return IN_TABLE(wide, cp) ? 2 : 1; +} + +static inline bool unicode_extended_pictographic(uint32_t cp) +{ + return IN_TABLE(extended_pictographic, cp); +} + +static inline bool unicode_emoji_presentation(uint32_t cp) +{ + return IN_TABLE(emoji_presentation, cp); +} + +static inline bool unicode_spacing_mark(uint32_t cp) +{ + return IN_TABLE(spacing_mark, cp); +} + +#undef IN_TABLE + +#endif diff --git a/meson.build b/meson.build new file mode 100644 index 0000000..a78fa68 --- /dev/null +++ b/meson.build @@ -0,0 +1,48 @@ +project('term', 'c', + version: '0.1.0', + meson_version: '>=1.1.0', + default_options: [ + 'c_std=c11', + 'warning_level=3', + 'b_ndebug=if-release', + ], +) + +cc = meson.get_compiler('c') +m_dep = cc.find_library('m', required: false) +if get_option('arch_flag') != '' + add_project_arguments(get_option('arch_flag'), language: 'c') +endif + +# A host sees include/ only. The sources also find the public headers +# without the term/ prefix. +term_inc = include_directories('include') +term_private_inc = include_directories('src', 'include/term') + +subdir('src') + +term_dep = declare_dependency( + include_directories: term_inc, + link_with: term_headless_lib.get_static_lib(), + dependencies: m_dep, +) +meson.override_dependency('term', term_dep) + +# The private headers too. Only the term tests and benchmarks use it. +term_internal_dep = declare_dependency( + include_directories: [term_inc, term_private_inc], + link_with: term_headless_lib.get_static_lib(), + dependencies: m_dep, +) + +# The short forms of term.h that the term tests use. A host project can use +# them in its own tests. +term_test_dep = declare_dependency( + include_directories: include_directories('tests/helpers'), + dependencies: term_dep, +) + +if get_option('tests') + subdir('tests') + subdir('benchmarks') +endif diff --git a/meson.options b/meson.options new file mode 100644 index 0000000..27bc018 --- /dev/null +++ b/meson.options @@ -0,0 +1,25 @@ +option( + 'arch_flag', + type: 'string', + value: '', + description: 'Compiler flag for the lowest instruction set, such as -march=x86-64-v3. Higher SIMD levels are picked at run time', +) +option( + 'tests', + type: 'boolean', + value: true, + description: 'Build and register the term tests and benchmarks', +) +option( + 'write_fuzz_seeds', + type: 'integer', + min: 1, + value: 1000, + description: 'Seed count for write-fuzz-test', +) +option( + 'kitty_tests', + type: 'boolean', + value: false, + description: 'Build the Linux Kitty parser compatibility tests (use an unsanitized build)', +) diff --git a/src/cell.h b/src/cell.h new file mode 100644 index 0000000..41b16eb --- /dev/null +++ b/src/cell.h @@ -0,0 +1,46 @@ +#ifndef TERM_CELL_H +#define TERM_CELL_H + +/* + * The cell layout that the terminal stores. Only src/term uses it. A + * snapshot converts these cells to the public struct term_cell. + */ + +#include "term.h" + +enum cell_flags { + CELL_BOLD = 1 << 0, + CELL_ITALIC = 1 << 1, + CELL_UNDERLINE = 1 << 2, + CELL_STRIKE = 1 << 3, + CELL_OVERLINE = 1 << 4, + CELL_EMOJI = 1 << 5, + CELL_TEXT = 1 << 6, + CELL_WIDE = 1 << 7, + CELL_TRAIL = 1 << 8, + CELL_WRAP_PAD = 1 << 9, + CELL_FAINT = 1 << 10, + CELL_DOUBLE_UNDERLINE = 1 << 11, + CELL_CURLY_UNDERLINE = 1 << 12, + CELL_DOTTED_UNDERLINE = 1 << 13, + CELL_DASHED_UNDERLINE = 1 << 14, + CELL_LINK = 1 << 15, +}; + +#define CELL_ANY_UNDERLINE \ + (CELL_UNDERLINE | CELL_DOUBLE_UNDERLINE | CELL_CURLY_UNDERLINE | \ + CELL_DOTTED_UNDERLINE | CELL_DASHED_UNDERLINE) + +/* c is a code point or a grapheme handle. pad holds the OSC 8 link id. */ +struct cell { + uint32_t c; + uint32_t fg; + uint32_t bg; + uint16_t flags; + uint16_t pad; +}; + +_Static_assert(sizeof(struct cell) == 16, + "struct cell must stay 16 bytes (4 cells per AVX-512 register)"); + +#endif diff --git a/src/compat.h b/src/compat.h new file mode 100644 index 0000000..d1264fe --- /dev/null +++ b/src/compat.h @@ -0,0 +1,171 @@ +#ifndef TERM_COMPAT_H +#define TERM_COMPAT_H + +/* + * Compiler and libc differences, kept in one place. GCC and Clang use their + * builtins. MSVC uses its intrinsics. Any other compiler, and any build with + * TERM_NO_COMPILER_BUILTINS, uses plain C. Plain C gives the same results, + * only slower. + */ + +#include <stdbool.h> +#include <stddef.h> +#include <stdint.h> +#include <stdlib.h> +#include <string.h> + +#if defined(__GNUC__) || defined(__clang__) +#define TERM_COMPILER_GNU 1 +#elif defined(_MSC_VER) +#define TERM_COMPILER_MSVC 1 +#include <intrin.h> +#endif + +#if !defined(TERM_NO_COMPILER_BUILTINS) && defined(TERM_COMPILER_GNU) +#define TERM_HAVE_BUILTINS 1 +#elif !defined(TERM_NO_COMPILER_BUILTINS) && defined(TERM_COMPILER_MSVC) +#define TERM_HAVE_MSVC_BITOPS 1 +#endif + +#if defined(TERM_COMPILER_GNU) +#define TERM_HOT __attribute__((hot)) +#define TERM_NOINLINE __attribute__((noinline)) +#define TERM_ALWAYS_INLINE __attribute__((always_inline)) +#define TERM_MAYBE_UNUSED __attribute__((unused)) +#elif defined(TERM_COMPILER_MSVC) +#define TERM_HOT +#define TERM_NOINLINE __declspec(noinline) +#define TERM_ALWAYS_INLINE __forceinline +#define TERM_MAYBE_UNUSED +#else +#define TERM_HOT +#define TERM_NOINLINE +#define TERM_ALWAYS_INLINE +#define TERM_MAYBE_UNUSED +#endif + +#if defined(TERM_HAVE_BUILTINS) +#define TERM_EXPECT(x, v) __builtin_expect((x), (v)) +#else +#define TERM_EXPECT(x, v) (x) +#endif + +/* term_ctz32(0) is 32 and term_ctz64(0) is 64 on every path. */ +#if defined(TERM_HAVE_BUILTINS) + +static inline int term_ctz32(uint32_t x) { return x ? __builtin_ctz(x) : 32; } +static inline int term_popcount32(uint32_t x) { return __builtin_popcount(x); } +static inline int term_ctz64(uint64_t x) { return x ? __builtin_ctzll(x) : 64; } +static inline int term_popcount64(uint64_t x) { return __builtin_popcountll(x); } + +#else + +#if defined(TERM_HAVE_MSVC_BITOPS) +static inline int term_ctz32(uint32_t x) +{ + unsigned long i; + return _BitScanForward(&i, x) ? (int)i : 32; +} +static inline int term_popcount32(uint32_t x) { return (int)__popcnt(x); } +#else +static inline int term_ctz32(uint32_t x) +{ + if (!x) + return 32; + int n = 0; + while (!(x & 1u)) { + x >>= 1; + n++; + } + return n; +} +static inline int term_popcount32(uint32_t x) +{ + int n = 0; + for (; x; x &= x - 1) + n++; + return n; +} +#endif + +/* Without builtins, the 64-bit forms are built from the 32-bit ones. */ +static inline int term_ctz64(uint64_t x) +{ + uint32_t lo = (uint32_t)x; + return lo ? term_ctz32(lo) : 32 + term_ctz32((uint32_t)(x >> 32)); +} + +static inline int term_popcount64(uint64_t x) +{ + return term_popcount32((uint32_t)x) + term_popcount32((uint32_t)(x >> 32)); +} + +#endif + +/* A spin-wait hint. Compilers without one get a no-op. */ +static inline void term_cpu_relax(void) +{ +#if defined(TERM_COMPILER_GNU) +#if defined(TERM_HAVE_BUILTINS) && defined(__x86_64__) + __builtin_ia32_pause(); +#elif defined(__aarch64__) + __asm__ __volatile__("yield"); +#endif +#elif defined(TERM_COMPILER_MSVC) +#if defined(_M_X64) || defined(_M_IX86) + _mm_pause(); +#elif defined(_M_ARM64) + __yield(); +#endif +#endif +} + +/* strdup is POSIX, so the core copies strings itself. */ +static inline char *term_strdup(const char *s) +{ + size_t n = strlen(s) + 1; + char *copy = malloc(n); + if (copy) + memcpy(copy, s, n); + return copy; +} + +/* ASCII-only prefix test, as strncasecmp() does in the C locale. */ +static inline bool term_has_prefix_ci(const char *s, const char *prefix) +{ + for (; *prefix; prefix++, s++) { + unsigned char a = (unsigned char)*s; + unsigned char b = (unsigned char)*prefix; + if (a >= 'A' && a <= 'Z') + a += 'a' - 'A'; + if (b >= 'A' && b <= 'Z') + b += 'a' - 'A'; + if (a != b) + return false; + } + return true; +} + +/* Allocates size bytes that start on an align-byte boundary. align must be a + * power of two. The raw pointer is stored just before the block, so + * term_aligned_free() can release it. C11 aligned_alloc() and POSIX + * posix_memalign() are not on every toolchain. */ +static inline void *term_aligned_alloc(size_t align, size_t size) +{ + if (size > SIZE_MAX - align - sizeof(void *)) + return NULL; + void *raw = malloc(size + align + sizeof(void *)); + if (!raw) + return NULL; + uintptr_t p = ((uintptr_t)raw + sizeof(void *) + align - 1) & ~((uintptr_t)align - 1); + ((void **)p)[-1] = raw; + return (void *)p; +} + +static inline void term_aligned_free(void *p) +{ + if (p) + free(((void **)p)[-1]); +} + +#endif diff --git a/src/core.c b/src/core.c new file mode 100644 index 0000000..5d50db9 --- /dev/null +++ b/src/core.c @@ -0,0 +1,1311 @@ +#include "compat.h" +#include "core.h" +#include "kernels.h" +#include "pen.h" +#include "screen.h" +#include "text_run.h" +#include "unicode/grapheme.h" +#include "unicode/unicode.h" +#include "unicode/utf8.h" + +#include <stdlib.h> +#include <string.h> +#include <stdio.h> + +static uint16_t cluster_presentation(const uint32_t *cps, int len) +{ + bool pictographic = unicode_extended_pictographic(cps[0]); + bool emoji = false; + for (int i = 1; i < len; i++) { + if (cps[i] == 0xFE0F) + return CELL_EMOJI; + if (cps[i] == 0xFE0E) + return CELL_TEXT; + if (cps[i] == 0x20E3 || grapheme_regional_indicator(cps[i]) || + (pictographic && (cps[i] == 0x200D || grapheme_emoji_modifier(cps[i]) || + (cps[i] >= 0xE0020 && cps[i] <= 0xE007F)))) + emoji = true; + } + return emoji ? CELL_EMOJI : 0; +} + +void term_sc_print_glyph(struct term *t, uint32_t cp, int width, uint16_t flags) +{ + uint32_t base = cp; + if (TERM_EXPECT(grapheme_is_handle(cp), 0)) { + uint32_t cps[GRAPHEME_MAX_LEN]; + int len = term_gstore_expand(t->gstore, cp, cps, GRAPHEME_MAX_LEN); + base = cps[0]; + flags = (flags & ~(CELL_EMOJI | CELL_TEXT)) | cluster_presentation(cps, len); + } + if (TERM_EXPECT(width <= 0, 0)) + width = term_text_class(base); + if (width == 2 || (flags & CELL_EMOJI)) + width = 2; + else + width = 1; + if (TERM_EXPECT(flags & CELL_TEXT, 0)) + width = 1; + + if (TERM_EXPECT(width == 1 && !t->pending_wrap && !t->insert_mode && + !t->lr_margin_mode && t->cursor_x < t->cols, 1)) { + struct srow *srow = term_row(t, t->cursor_y); + int cx = t->cursor_x; + int len = srow->len; + if (TERM_EXPECT(srow->kind == ROW_CELLS && cx <= len, 1)) { + struct cell *row = srow->cells; + if (TERM_EXPECT(!((cx < len && (row[cx].flags & CELL_TRAIL)) || + (cx + 1 < len && (row[cx + 1].flags & CELL_TRAIL))), 1)) { + row[cx] = (struct cell){cp, + t->inverse ? t->cur_bg : t->cur_fg, + t->inverse ? t->cur_fg : t->cur_bg, + flags, t->cur_link}; + t->last_char = cp; + cx++; + if (cx > len) srow->len = (uint16_t)cx; + srow->version++; + if (cx < t->cols) { + t->cursor_x = cx; + t->pending_wrap = false; + } else { + t->cursor_x = t->cols - 1; + t->pending_wrap = t->autowrap; + } + return; + } + } + } + + if (TERM_EXPECT(t->pending_wrap, 0)) { + row_set_wrapped(term_row(t, t->cursor_y), true); + t->cursor_x = t->lr_margin_mode ? t->left_margin : 0; + sc_linefeed(t, 1); + t->pending_wrap = false; + } + + int p = phys_row(t, t->cursor_y); + struct cell *row = row_cells(t, p); + struct srow *srow = &t->scr.slot[p]; + int cx = t->cursor_x; + int cols = t->cols; + int right = t->lr_margin_mode ? t->right_margin : cols - 1; + + if (TERM_EXPECT(width == 2 && cols > 1 && cx == right && t->autowrap, 0)) { + row_fill_gap(t, p, row, cx); + clear_wide_edges(row, cx, right + 1, srow->len); + row[cx] = (struct cell){ ' ', TERM_COLOR_FG, TERM_COLOR_BG, CELL_WRAP_PAD, 0 }; + srow->len = (uint16_t)cols; + srow->version++; + row_set_wrapped(srow, true); + t->cursor_x = t->lr_margin_mode ? t->left_margin : 0; + sc_linefeed(t, 1); + t->pending_wrap = false; + p = phys_row(t, t->cursor_y); + row = row_cells(t, p); + srow = &t->scr.slot[p]; + cx = t->cursor_x; + } + + int span = width > right + 1 - cx ? 1 : width; + if (span == 2) { + flags |= CELL_WIDE; + srow->flags |= RF_WIDE; + } else { + flags &= ~CELL_WIDE; + } + row_fill_gap(t, p, row, cx); + if (TERM_EXPECT(t->insert_mode, 0)) { + if (cx == srow->len) + row[cx] = (struct cell){ ' ', TERM_COLOR_FG, TERM_COLOR_BG, 0, 0 }; + int move = right + 1 - cx - span; + if (move > 0) + memmove(&row[cx + span], &row[cx], (size_t)move * sizeof(struct cell)); + int new_len = srow->len + span; + srow->len = (uint16_t)(new_len > cols ? cols : new_len); + repair_wide_row(row, right, right + 1, cols, srow->len); + } + clear_wide_edges(row, cx, cx + span, srow->len); + + uint32_t fg = t->inverse ? t->cur_bg : t->cur_fg; + uint32_t bg = t->inverse ? t->cur_fg : t->cur_bg; + write_cell_span(row, cx, (struct cell){ cp, fg, bg, flags, t->cur_link }, span); + if (cx + span > srow->len) + srow->len = (uint16_t)(cx + span); + srow->version++; + t->last_char = cp; + + if (cx + span <= right) { + t->cursor_x = cx + span; + t->pending_wrap = false; + } else { + t->cursor_x = right; + t->pending_wrap = t->autowrap; + } +} + +static inline struct cell *last_written_cell(struct term *t) +{ + int right = (t->lr_margin_mode && t->cursor_x <= t->right_margin) + ? t->right_margin : t->cols - 1; + int x = t->pending_wrap ? right : t->cursor_x - 1; + if (t->lr_margin_mode && x < t->left_margin) + return NULL; + + struct srow *srow = term_row(t, t->cursor_y); + if (x < 0 || x >= srow->len) + return NULL; + int p = phys_row(t, t->cursor_y); + struct cell *row = row_cells(t, p); + if (x > 0 && (row[x].flags & CELL_TRAIL)) + x--; + return &row[x]; +} + +bool term_sc_extend_cluster(struct term *t, uint32_t cp, int klass) +{ + if (!can_extend_cluster(t->last_char, cp, klass)) + return false; + + struct cell *prev = last_written_cell(t); + uint32_t cps[GRAPHEME_MAX_LEN]; + int len = prev && prev->c > ' ' ? term_gstore_expand(t->gstore, prev->c, cps, GRAPHEME_MAX_LEN) : 0; + if (!len) + return joins_cluster(cp, klass); + if (!cluster_joins(cps, len, cp, klass)) + return false; + if (len >= GRAPHEME_MAX_LEN) + return true; + cps[len++] = cp; + + struct cell cell = *prev; + int p = phys_row(t, t->cursor_y); + struct cell *row = row_cells(t, p); + int x = (int)(prev - row); + int span = (cell.flags & CELL_WIDE) && x + 1 < t->cols ? 2 : 1; + struct srow *srow = &t->scr.slot[p]; + if (srow->len == x + span) { + fill_cells(prev, span, TERM_COLOR_FG, TERM_COLOR_BG); + srow->len = (uint16_t)x; + } else { + fill_cells(prev, span, cell.fg, cell.bg); + } + t->cursor_x = x; + t->pending_wrap = false; + t->last_char = term_gstore_intern(t->gstore, cps, len); + term_sc_print_glyph(t, t->last_char, -1, + cell.flags & ~(CELL_WIDE | CELL_EMOJI | CELL_TEXT)); + + prev = last_written_cell(t); + if (prev) { + row = row_cells(t, phys_row(t, t->cursor_y)); + prev->fg = cell.fg; + prev->bg = cell.bg; + if ((prev->flags & CELL_WIDE) && prev < row + t->cols - 1) { + prev[1].fg = cell.fg; + prev[1].bg = cell.bg; + } + } + return true; +} + +/* n blank default rows scroll off the top, after the screen rows. */ +static void push_blank_lines(struct term *t, size_t n) +{ + if (screen_has_history(t)) { + term_hist_push_blank(&t->hist, n); + } else { + t->scr.top_id += n; + t->scr.top_seg = 0; + } +} + +static void scroll_up_bce(struct term *t, size_t count) +{ + size_t rows = (size_t)t->rows; + if (t->alternate || count <= rows) { + for (size_t i = count < rows ? count : rows; i > 0; i--) + term_screen_scroll_up(t); + return; + } + size_t rem = count - rows; + if (!using_default_colors(t)) { + /* Rows past the history limit would leave again at once. */ + size_t keep = screen_has_history(t) ? t->hist.line_limit : 0; + for (size_t i = rows + (rem < keep ? rem : keep); i > 0; i--) + term_screen_scroll_up(t); + return; + } + bool skip = screen_has_history(t) && rem >= t->hist.line_limit; + if (!skip && !hist_is_open(&t->hist)) { + skip = true; + for (size_t y = 0; y < rows && skip; y++) + skip = t->scr.slot[y].len == 0 && !(t->scr.slot[y].flags & RF_WRAPPED); + } + if (skip) { + /* The screen is blank, or all of it leaves history again. */ + for (size_t y = 0; y < rows; y++) + screen_new_row(t, &t->scr.slot[y]); + push_blank_lines(t, count); + } else { + for (size_t i = 0; i < rows; i++) + term_screen_scroll_up(t); + push_blank_lines(t, rem); + } + if (t->prompt_y > 0) + t->prompt_y = 0; + clamp_selection_after_scroll(t); + t->full_gen++; +} + +void term_sc_linefeed_slow(struct term *t, size_t n) +{ + if (n == 0) + return; + t->pending_wrap = false; + term_row(t, t->cursor_y)->version++; + + if (t->cursor_y > t->scroll_bottom) { + if (t->cursor_y + (int)n < t->rows) + t->cursor_y += (int)n; + else + t->cursor_y = t->rows - 1; + return; + } + + if (t->cursor_y + (int)n <= t->scroll_bottom) { + t->cursor_y += (int)n; + return; + } + + size_t scroll = (size_t)(t->cursor_y + (int)n - t->scroll_bottom); + t->cursor_y = t->scroll_bottom; + + if (TERM_EXPECT(t->scroll_top == 0 && t->scroll_bottom == t->rows - 1, 1)) { + if (TERM_EXPECT(scroll == 1, 1)) + term_screen_scroll_up(t); + else + scroll_up_bce(t, scroll); + } else if (t->scroll_top > 0 || t->alternate) { + int height = t->scroll_bottom - t->scroll_top + 1; + if (height <= 0) height = 1; + int count = scroll > (size_t)height ? height : (int)scroll; + term_screen_scroll_region(t, t->scroll_top, t->scroll_bottom, count, false, false); + } else { + int height = t->scroll_bottom - t->scroll_top + 1; + if (height <= 0) height = 1; + if (scroll <= (size_t)height) { + term_screen_scroll_region(t, t->scroll_top, t->scroll_bottom, (int)scroll, false, true); + } else { + term_screen_scroll_region(t, t->scroll_top, t->scroll_bottom, height, false, true); + size_t rem = scroll - (size_t)height; + if (using_default_colors(t)) { + push_blank_lines(t, rem); + clamp_selection_after_scroll(t); + } else { + size_t cap = t->hist.line_limit > 0 ? t->hist.line_limit : 1; + if (rem > cap) + rem = cap; + while (rem > 0) { + int chunk = rem > (size_t)height ? height : (int)rem; + term_screen_scroll_region(t, t->scroll_top, t->scroll_bottom, chunk, false, true); + rem -= (size_t)chunk; + } + } + } + } +} + +void term_sc_tab(struct term *t, int n) +{ + int dir = n >= 0 ? 1 : -1; + int count = n >= 0 ? n : -n; + if (count > t->cols) count = t->cols; + while (count-- > 0) { + int x = t->cursor_x + dir; + while (x > 0 && x < t->cols - 1 && !t->tabs[x]) + x += dir; + t->cursor_x = x < 0 ? 0 : x >= t->cols ? t->cols - 1 : x; + } + t->pending_wrap = false; + struct srow *cur = term_row(t, t->cursor_y); + if (cur) cur->version++; +} + +void term_sc_scroll_region(struct term *t, int top, int bottom, int n, bool down) +{ + t->pending_wrap = false; + term_screen_scroll_region(t, top, bottom, n, down, false); +} + +void term_sc_scroll(struct term *t, int n, bool down) +{ + t->pending_wrap = false; + term_screen_scroll_region(t, t->scroll_top, t->scroll_bottom, n, down, true); +} + +#define CELL_SGR_FLAGS (CELL_BOLD | CELL_FAINT | CELL_ITALIC | \ + CELL_STRIKE | CELL_OVERLINE | CELL_ANY_UNDERLINE) + +static inline void sgr_cell_reset(struct cell *cell) +{ + cell->fg = TERM_COLOR_FG; + cell->bg = TERM_COLOR_BG; + cell->flags &= (uint16_t)~CELL_SGR_FLAGS; +} + +static inline void sgr_cell_underline(struct cell *cell, int style) +{ + if (style < 0 || style > 5) style = 1; + cell->flags = (uint16_t)((cell->flags & ~CELL_ANY_UNDERLINE) | + term_ul_flags[style]); +} + +static void apply_sgr_to_cell(struct cell *cell, const struct csi *csi, int first) +{ + if (first >= csi->nparams) { + sgr_cell_reset(cell); + return; + } + for (int i = first; i < csi->nparams; i++) { + int v = csi->params[i]; + uint32_t color; + bool ok; + switch (v) { + case 0: sgr_cell_reset(cell); break; + case 1: cell->flags |= CELL_BOLD; break; + case 2: cell->flags |= CELL_FAINT; break; + case 3: cell->flags |= CELL_ITALIC; break; + case 4: { + int style = 1; + if (i + 1 < csi->nparams && (csi->subparams & (1u << (i + 1)))) + style = csi->params[++i]; + sgr_cell_underline(cell, style); + break; + } + case 7: { uint32_t tmp = cell->fg; cell->fg = cell->bg; cell->bg = tmp; break; } + case 9: cell->flags |= CELL_STRIKE; break; + case 21: sgr_cell_underline(cell, 2); break; + case 22: cell->flags &= (uint16_t)~(CELL_BOLD | CELL_FAINT); break; + case 23: cell->flags &= (uint16_t)~CELL_ITALIC; break; + case 24: sgr_cell_underline(cell, 0); break; + case 29: cell->flags &= (uint16_t)~CELL_STRIKE; break; + case 38: + i += sgr_extended_color(csi->params, csi->nparams, csi->subparams, i, &color, &ok); + if (ok) cell->fg = color; + break; + case 39: cell->fg = TERM_COLOR_FG; break; + case 48: + i += sgr_extended_color(csi->params, csi->nparams, csi->subparams, i, &color, &ok); + if (ok) cell->bg = color; + break; + case 49: cell->bg = TERM_COLOR_BG; break; + case 53: cell->flags |= CELL_OVERLINE; break; + case 55: cell->flags &= (uint16_t)~CELL_OVERLINE; break; + case 58: + i += sgr_extended_color(csi->params, csi->nparams, csi->subparams, i, &color, &ok); + break; + default: + if ((unsigned)(v - 30) < 8) cell->fg = term_color_index(v - 30); + else if ((unsigned)(v - 40) < 8) cell->bg = term_color_index(v - 40); + else if ((unsigned)(v - 90) < 8) cell->fg = term_color_index(v - 90 + 8); + else if ((unsigned)(v - 100) < 8) cell->bg = term_color_index(v - 100 + 8); + break; + } + } +} + +static void deccara_row(struct term *t, int y, int x0, int x1, const struct csi *csi) +{ + int p = phys_row(t, y); + struct cell *row = row_cells(t, p); + struct srow *srow = term_row(t, y); + int len = srow->len; + if (x0 > 0 && x0 < len && (row[x0].flags & CELL_TRAIL)) x0--; + if (x1 + 1 < t->cols && x1 < len && (row[x1].flags & CELL_WIDE)) x1++; + term_row_fill_gap(srow, x1 + 1); + for (int x = x0; x <= x1; x++) + apply_sgr_to_cell(&row[x], csi, 4); + srow->version++; +} + +void term_sc_deccara(struct term *t, const struct csi *csi) +{ + int top = csi->params[0] > 0 ? csi->params[0] - 1 : 0; + int left = csi->nparams > 1 && csi->params[1] > 0 ? csi->params[1] - 1 : 0; + int bottom = csi->nparams > 2 && csi->params[2] > 0 ? csi->params[2] - 1 : t->rows - 1; + int right = csi->nparams > 3 && csi->params[3] > 0 ? csi->params[3] - 1 : t->cols - 1; + if (top > bottom || left > right || top >= t->rows || left >= t->cols) return; + if (bottom >= t->rows) bottom = t->rows - 1; + if (right >= t->cols) right = t->cols - 1; + if (t->decsace == 2) { + for (int y = top; y <= bottom; y++) + deccara_row(t, y, left, right, csi); + } else { + deccara_row(t, top, left, top == bottom ? right : t->cols - 1, csi); + for (int y = top + 1; y < bottom; y++) + deccara_row(t, y, 0, t->cols - 1, csi); + if (bottom > top) + deccara_row(t, bottom, 0, right, csi); + } +} + +void term_sc_decaln(struct term *t) +{ + struct cell fill = { .c = 'E', .fg = TERM_COLOR_FG, .bg = TERM_COLOR_BG }; + for (int y = 0; y < t->rows; y++) { + int p = phys_row(t, y); + struct cell *row = row_cells(t, p); + for (int x = 0; x < t->cols; x++) row[x] = fill; + struct srow *srow = &t->scr.slot[p]; + srow->len = (uint16_t)t->cols; + srow->flags = RF_TAIL_PADDED; + srow->kind = ROW_CELLS; + srow->version++; + } + sc_move(t, 0, 0); +} + +void term_sc_reset(struct term *t) +{ + t->cursor_x = t->cursor_y = 0; + t->saved = term_saved_cursor_initial(); + t->prompt_y = -1; + t->has_prompt = false; + t->graphics = t->pending_wrap = false; + t->last_char = 0; + t->charsets[0] = t->charsets[1] = 'B'; + t->charset_active = 0; + term_tabs_clear(t); + term_tabs_default(t); + term_screen_erase_all(t); + t->full_gen++; +} + +void term_sc_set_reverse_video(struct term *t, bool enabled) +{ + if (t->reverse_video != enabled) { + t->reverse_video = enabled; + t->full_gen++; + } +} + +void term_sc_set_autowrap(struct term *t, bool enabled) +{ + t->autowrap = enabled; + t->pending_wrap = false; +} + +void term_sc_set_cursor_style(struct term *t, int ps) +{ + if (ps == 0) { + t->cursor_style = t->default_cursor_style; + t->cursor_blink = t->default_cursor_blink; + } else if (ps >= 1 && ps <= 6) { + t->cursor_style = ps < 3 ? TERM_CURSOR_BLOCK + : ps < 5 ? TERM_CURSOR_UNDERLINE : TERM_CURSOR_BAR; + t->cursor_blink = (ps & 1); + } + term_row(t, t->cursor_y)->version++; +} + +static inline int newline_bytes(const char *buf, size_t len) +{ + if (len >= 2 && buf[0] == '\r' && buf[1] == '\n') return 2; + if (len >= 1 && buf[0] == '\n') return 1; + return 0; +} + +static inline bool next_byte_wraps(const char *buf, size_t len, size_t pos, uint32_t last) +{ + if (pos >= len) return false; + uint32_t cp; + if (!utf8_decode_one(buf + pos, len - pos, &cp)) return false; + int w = term_text_class(cp); + return w > 0 && !can_extend_cluster(last, cp, w); +} + +static void flush_orig_rows(struct term *t, struct screen *s, + int *orig_y, int *orig_rem, int count) +{ + int y = *orig_y; + int end = y + count; + *orig_y = end; + *orig_rem -= count; + struct hist_ascii_line q[64]; + while (y < end) { + int p = s->top + y; + struct srow *first = &s->slot[p >= s->rows ? p - s->rows : p]; + uintptr_t begin = (uintptr_t)t->ascii_tail.data; + uintptr_t limit = begin + t->ascii_tail.cap; + uintptr_t base = (uintptr_t)first->text.text; + if (first->kind == ROW_ASCII && base >= begin && base < limit) { + /* The LF stream stores checked lines here, including each LF. + * A read or edit changes the row kind before it can change text. */ + uint16_t ends[64 + 65]; + ends[0] = 0; + int span = 0, lines = 0, closed = y; + for (int j = y; j < end && j - y < 64; j++) { + int p = s->top + j; + struct srow *row = &s->slot[p >= s->rows ? p - s->rows : p]; + bool wrapped = (row->flags & RF_WRAPPED) != 0; + size_t size = (size_t)row->len + !wrapped; + if (row->kind != ROW_ASCII || (uintptr_t)row->text.text != base + span || + size > limit - base - span || size + span > UINT16_MAX) break; + span += row->len; + if (!wrapped) { + ends[++lines] = (uint16_t)++span; + closed = j + 1; + } + } + if (lines) { + term_hist_push_ascii_lf(&t->hist, (const char *)base, ends, lines); + if (t->view.anchor.active) term_hist_view_note_push(&t->view, &t->hist, t->cols); + y = closed; + continue; + } + } + int n = 0; + while (y + n < end && n < 64) { + int p = s->top + y + n; + struct srow *row = &s->slot[p >= s->rows ? p - s->rows : p]; + struct text_row *tr = &row->text; + bool wrapped = (row->flags & RF_WRAPPED) != 0; + if (wrapped && row->len < s->cols) break; + if (row->kind == ROW_TEXT) { + if (!(tr->flags & HLINE_NARROW) || tr->runs.len || tr->fill_to > tr->cells || + !pen_attr_is_default(&tr->start) || !pen_attr_is_default(&tr->runs.pen)) break; + } else if (row->kind != ROW_ASCII && row->len) break; + q[n++] = (struct hist_ascii_line){ + .ptr = (const char *)tr->text, .len = row->len, .wrapped = wrapped, + }; + } + if (n) { + term_hist_push_ascii_lines(&t->hist, q, n, NULL); + if (t->view.anchor.active) term_hist_view_note_push(&t->view, &t->hist, t->cols); + y += n; + } else term_screen_push_row(t, screen_get_row(s, y++)); + } +} + +static void flush_stream_queue(struct term *t, struct screen *s, + struct hist_ascii_line *q, int *q_count, int q_cap, + int *orig_y, int *orig_rem, size_t *flushed_scrolled, + const struct pen_attr *attr, bool def_pen) +{ + if (*orig_rem > 0) + flush_orig_rows(t, s, orig_y, orig_rem, *orig_rem); + int flush_n = *q_count - q_cap; + (void)def_pen; + term_hist_push_ascii_lines(&t->hist, q, flush_n, attr); + if (q_cap > 0) { + if (flush_n >= q_cap) + memcpy(q, q + flush_n, (size_t)q_cap * sizeof(q[0])); + else + memmove(q, q + flush_n, (size_t)q_cap * sizeof(q[0])); + } + *q_count = q_cap; + *flushed_scrolled += (size_t)flush_n; +} + +static inline void emit_stream_bulk_wrapped(struct term *t, struct screen *s, + const char *src, int bulk_lines, int cols, + const struct pen_attr *attr, bool def_pen, + struct hist_ascii_line *q, int q_cap, + int *q_count, int *orig_y, int *orig_rem, + size_t *flushed_scrolled) +{ + if (TERM_EXPECT(t->pending_wrap, 0)) { + row_set_wrapped(screen_get_row(s, s->rows - 1), true); + t->pending_wrap = false; + t->cursor_x = 0; + term_screen_scroll_up(t); + } + if (bulk_lines < q_cap) { + const char *p = src; + for (int r = 0; r < bulk_lines; r++, p += cols) { + if (*q_count == 256) { + flush_stream_queue(t, s, q, q_count, q_cap, orig_y, orig_rem, + flushed_scrolled, attr, def_pen); + } + q[(*q_count)++] = (struct hist_ascii_line){ p, (uint16_t)cols, 1 }; + } + return; + } + + if (*orig_rem > 0) + flush_orig_rows(t, s, orig_y, orig_rem, *orig_rem); + if (*q_count > 0) { + term_hist_push_ascii_lines(&t->hist, q, *q_count, attr); + *flushed_scrolled += (size_t)*q_count; + *q_count = 0; + } + int hist_rows = bulk_lines - q_cap; + if (hist_rows > 0) { + term_hist_push_ascii(&t->hist, src, hist_rows * cols, attr, true); + *flushed_scrolled += (size_t)hist_rows; + } + const char *sp = src + (size_t)hist_rows * (size_t)cols; + for (int r = 0; r < q_cap; r++, sp += cols) + q[r] = (struct hist_ascii_line){ sp, (uint16_t)cols, 1 }; + *q_count = q_cap; +} + +static TERM_NOINLINE size_t stream_ascii_wrapped(struct term *t, const char *buf, size_t len) +{ + int limit = len > INT32_MAX ? INT32_MAX : (int)len; + int run = scan_ascii_run(buf, limit); + if (!run) return 0; + int cols = t->cols; + int count = (run - 1) / cols; + if (!count) return 0; + struct screen *s = &t->scr; + int rows = s->rows, keep = rows - 1; + int scrolls = count + t->pending_wrap; + int old_rows = keep + t->pending_wrap; + if (t->pending_wrap) { + row_set_wrapped(screen_get_row(s, rows - 1), true); + t->pending_wrap = false; + t->cursor_x = 0; + } + int old_count = scrolls < old_rows ? scrolls : old_rows; + int orig_y = 0, orig_rem = old_rows; + flush_orig_rows(t, s, &orig_y, &orig_rem, old_count); + int extra = count - (keep - orig_rem); + if (extra) term_hist_push_ascii(&t->hist, buf, extra * cols, NULL, true); + s->top = (s->top + scrolls % rows) % rows; + const char *src = buf + extra * cols; + uint8_t *copy_dst = NULL; + const char *copy_src = src; + size_t copy_len = 0; + uint64_t uid = t->next_row_uid; + for (int y = orig_rem; y < rows - 1; y++, src += cols) { + int p = s->top + y; + struct srow *dst = &s->slot[p >= rows ? p - rows : p]; + dst->uid = ++uid; + dst->version++; + dst->kind = ROW_ASCII; + dst->len = (uint16_t)cols; + dst->flags = RF_WRAPPED; + struct text_row *tr = &dst->text; + tr->text = dst->ascii; + if (copy_len && tr->text != copy_dst + copy_len) { + memcpy(copy_dst, copy_src, copy_len); + copy_len = 0; + } + if (!copy_len) { + copy_dst = tr->text; + copy_src = src; + } + copy_len += (size_t)cols; + } + if (copy_len) memcpy(copy_dst, copy_src, copy_len); + t->next_row_uid = uid; + screen_new_row(t, screen_get_row(s, rows - 1)); + if (t->prompt_y > 0) + t->prompt_y = t->prompt_y > scrolls ? t->prompt_y - scrolls : 0; + clamp_selection_after_scroll(t); + t->last_char = (unsigned char)buf[count * cols - 1]; + t->cursor_x = 0; + t->pending_wrap = false; + t->full_gen++; + return (size_t)count * cols; +} + +#if defined(TERM_KERNEL_AVX512) +/* Cut *count at the first line longer than half a history page. Check + * whole blocks of 32 lines. Return the line where the check stopped. */ +TERM_TARGET_AVX512 +static int long_lines_avx512(const uint16_t *ends, int *count) +{ + int j = 0; + for (; j + 32 <= *count; j += 32) { + __m512i start = _mm512_loadu_si512(ends + j); + __m512i end = _mm512_loadu_si512(ends + j + 1); + uint32_t bad = _mm512_cmpgt_epu16_mask(_mm512_sub_epi16(end, start), + _mm512_set1_epi16(HIST_PAGE_USABLE / 2)); + if (bad) { + *count = j + term_ctz32(bad); + break; + } + } + return j; +} +#endif + +static TERM_NOINLINE size_t stream_ascii_lf(struct term *t, const char *buf, size_t len) +{ + int limit = len < UINT16_MAX ? (int)len : UINT16_MAX; + size_t need = ((size_t)limit + 65) * sizeof(uint16_t); + if (need > t->ascii_ends.cap) { + void *data = realloc(t->ascii_ends.data, need); + if (!data) return 0; + t->ascii_ends.data = data; + t->ascii_ends.cap = need; + } + uint16_t *ends = t->ascii_ends.data; + int count = scan_ascii_lf(buf, limit, ends); + int j = 0; +#if defined(TERM_KERNEL_AVX512) + if (term_use_avx512()) + j = long_lines_avx512(ends, &count); +#endif + for (; j < count; j++) { + if (ends[j + 1] - ends[j] > HIST_PAGE_USABLE / 2) { + count = j; + break; + } + } + struct screen *s = &t->scr; + int cols = t->cols, rows = t->rows; + int keep = rows - 1, first = count; + int tail_rows = 0; + while (first > 0 && tail_rows < keep) { + first--; + tail_rows += (int)rows_narrow(ends[first + 1] - ends[first] - 1, cols); + } + if (!first) return 0; + int skip = (tail_rows - keep) * cols; + int source_start = ends[first] + skip; + size_t tail_bytes = (size_t)(ends[count] - source_start); + uint8_t *tail_data = t->ascii_tail.data; + if (tail_bytes > t->ascii_tail.cap) { + tail_data = malloc(tail_bytes); + if (!tail_data) return 0; + } + int orig_y = 0, orig_rem = rows - 1; + flush_orig_rows(t, s, &orig_y, &orig_rem, orig_rem); + if (tail_data != t->ascii_tail.data) { + free(t->ascii_tail.data); + t->ascii_tail.data = tail_data; + t->ascii_tail.cap = tail_bytes; + } + memcpy(tail_data, buf + source_start, tail_bytes); + term_hist_push_ascii_lf(&t->hist, buf, ends, first); + if (skip) + term_hist_push_ascii(&t->hist, buf + ends[first], skip, NULL, true); + uint32_t last = t->last_char; + for (int j = count; j > 0; j--) { + if (ends[j] - ends[j - 1] > 1) { + last = (unsigned char)buf[ends[j] - 2]; + break; + } + } + s->top = 0; + int y = 0; + uint64_t uid = t->next_row_uid; + for (int j = first; j < count; j++) { + int start = ends[j] + (j == first ? skip : 0); + int n = ends[j + 1] - 1 - start; + do { + int k = n > cols ? cols : n; + struct srow *dst = &s->slot[y++]; + dst->uid = ++uid; + dst->version++; + dst->flags = n > cols ? RF_WRAPPED : 0; + dst->len = (uint16_t)k; + dst->kind = ROW_ASCII; + dst->text.text = tail_data + start - source_start; + n -= k; + start += k; + } while (n > 0); + } + t->next_row_uid = uid; + screen_new_row(t, &s->slot[rows - 1]); + if (t->prompt_y > 0) t->prompt_y = 0; + clamp_selection_after_scroll(t); + t->last_char = last; + t->cursor_x = 0; + t->pending_wrap = false; + t->full_gen++; + return ends[count]; +} + +/* Bit i of the result is set when buf[i] is not printable ASCII. *nl gets + * the line feeds. Both cover 64 bytes. */ +#if defined(TERM_KERNEL_AVX512) +TERM_TARGET_AVX512 +static inline uint64_t line_masks_avx512(const char *buf, uint64_t *nl) +{ + __m512i v = _mm512_loadu_si512((const void *)buf); + *nl = _mm512_cmpeq_epi8_mask(v, _mm512_set1_epi8('\n')); + return _mm512_cmpge_epu8_mask(_mm512_sub_epi8(v, _mm512_set1_epi8(0x20)), + _mm512_set1_epi8(0x7f - 0x20)); +} +#endif + +#if defined(TERM_KERNEL_AVX2) +TERM_TARGET_AVX2 +static inline uint64_t line_masks_avx2(const char *buf, uint64_t *nl) +{ + uint64_t lf = 0, controls = 0; + for (int b = 0; b < 64; b += 32) { + __m256i v = _mm256_loadu_si256((const __m256i *)(buf + b)); + __m256i d = _mm256_sub_epi8(v, _mm256_set1_epi8(0x20)); + __m256i printable = _mm256_cmpeq_epi8( + _mm256_min_epu8(d, _mm256_set1_epi8(0x7f - 0x21)), d); + lf |= (uint64_t)(uint32_t)_mm256_movemask_epi8( + _mm256_cmpeq_epi8(v, _mm256_set1_epi8('\n'))) << b; + controls |= (uint64_t)(uint32_t)~_mm256_movemask_epi8(printable) << b; + } + *nl = lf; + return controls; +} +#endif + +static inline uint64_t line_masks(const char *buf, uint64_t *nl) +{ +#if defined(TERM_KERNEL_AVX2) + if (term_use_avx2()) + return line_masks_avx2(buf, nl); +#endif + uint64_t lf = 0, controls = 0; +#if defined(__aarch64__) + for (int b = 0; b < 64; b += 16) { + uint8x16_t v = vld1q_u8((const uint8_t *)buf + b); + lf |= (uint64_t)term_lane_bits(vceqq_u8(v, vdupq_n_u8('\n'))) << b; + controls |= (uint64_t)term_lane_bits(vcgeq_u8( + vsubq_u8(v, vdupq_n_u8(0x20)), vdupq_n_u8(0x7f - 0x20))) << b; + } +#else + for (int b = 0; b < 64; b++) { + unsigned char c = (unsigned char)buf[b]; + if (c == '\n') lf |= UINT64_C(1) << b; + if (c < 0x20 || c >= 0x7f) controls |= UINT64_C(1) << b; + } +#endif + *nl = lf; + return controls; +} + +/* One body for every level. isa is a constant in each copy below, so each + * copy keeps only its own kernels. */ +static inline TERM_ALWAYS_INLINE size_t bulk_ascii_lines_body(struct term *t, const char *buf, + size_t len, int isa) +{ + (void)isa; + struct screen *s = &t->scr; + int rows = s->rows; + if (TERM_EXPECT(rows > 256, 0)) + return 0; + + size_t i = 0; + int cols = t->cols; + uint32_t flags = term_flags_pad(t); + bool autowrap = t->autowrap; + bool lnm = term_lnm(t); + int last_char = (int)t->last_char; + size_t flushed_scrolled = 0; + + struct pen_attr attr = { + .fg = t->cur_fg, + .bg = t->cur_bg, + .flags = (uint16_t)flags, + .link = (uint16_t)(flags >> 16), + }; + bool def_pen = (t->cur_fg == TERM_COLOR_FG && t->cur_bg == TERM_COLOR_BG && flags == 0); + + if (t->pending_wrap && + (len == 0 || (unsigned char)buf[0] < 0x20 || (unsigned char)buf[0] >= 0x7f)) + return 0; + + struct hist_ascii_line q[256]; + int q_cap = rows - 1; + int q_count = 0; + int orig_y = 0; + int orig_rem = q_cap; + +#define EMIT_STREAM_ROW_FAST(row_ptr, row_len, row_wrapped) do { \ + if (TERM_EXPECT(q_count == 256, 0)) \ + flush_stream_queue(t, s, q, &q_count, q_cap, &orig_y, &orig_rem, \ + &flushed_scrolled, &attr, def_pen); \ + q[q_count++] = (struct hist_ascii_line){ \ + (row_ptr), (uint16_t)(row_len), (uint8_t)(row_wrapped) \ + }; \ +} while (0) + +#define EMIT_STREAM_ROW(row_ptr, row_len, row_wrapped) do { \ + if (TERM_EXPECT(t->pending_wrap, 0)) { \ + row_set_wrapped(screen_get_row(s, rows - 1), true); \ + t->pending_wrap = false; \ + t->cursor_x = 0; \ + term_screen_scroll_up(t); \ + } \ + EMIT_STREAM_ROW_FAST(row_ptr, row_len, row_wrapped); \ +} while (0) + + while (i < len) { + /* Scan each block once. Keep the unfinished line in the input buffer. */ + if (lnm && !t->pending_wrap) { + size_t scan = i; + while (scan + 64 <= len) { + uint64_t nl, controls; +#if defined(TERM_KERNEL_AVX512) + if (isa == TERM_ISA_AVX512) + controls = line_masks_avx512(buf + scan, &nl); + else +#endif + controls = line_masks(buf + scan, &nl); + if (controls != nl) + break; + while (nl) { + size_t end = scan + (size_t)term_ctz64(nl); + if (!autowrap && end - i > (size_t)cols) + goto narrow_done; + if (end > i) + last_char = (unsigned char)buf[end - 1]; + EMIT_STREAM_ROW_FAST(buf + i, (int)(end - i), false); + i = end + 1; + nl &= nl - 1; + } + scan += 64; + if (!autowrap && scan - i > (size_t)cols) + break; + if (scan - i > HIST_PAGE_USABLE / 2) + break; + } + } +narrow_done: + if (i >= len) + break; +#if defined(TERM_KERNEL_AVX512) + if (isa == TERM_ISA_AVX512 && TERM_EXPECT(!lnm && !t->pending_wrap, 1)) { + while (i + 64 <= len) { + uint64_t nl; + uint64_t mask = line_masks_avx512(buf + i, &nl); + + if (mask == 0) { + int total_avail = (int)(len - i); + int run = scan_ascii_run(buf + i, total_avail); + + int bulk_lines = run / cols - (run % cols == 0); + if (bulk_lines > 0 && autowrap && (i + run == len || ((unsigned char)buf[i + run] >= 0x20 && (unsigned char)buf[i + run] < 0x7f) || (run % cols == 0))) { + emit_stream_bulk_wrapped(t, s, buf + i, bulk_lines, cols, &attr, def_pen, + q, q_cap, &q_count, &orig_y, &orig_rem, + &flushed_scrolled); + size_t write_cells = (size_t)bulk_lines * (size_t)cols; + last_char = (unsigned char)buf[i + write_cells - 1]; + i += write_cells; + continue; + } + + int limit = run > cols ? cols : run; + int eol = 0; + if (i + limit < len && buf[i + limit] == '\n' && lnm) { + eol = 1; + } else if (i + limit + 1 < len && buf[i + limit] == '\r' && buf[i + limit + 1] == '\n') { + eol = 2; + } + + if (eol > 0) { + last_char = (unsigned char)buf[i + limit - 1]; + EMIT_STREAM_ROW_FAST(buf + i, limit, false); + i += (size_t)(limit + eol); + continue; + } + + bool wrapped = limit == cols && autowrap && + next_byte_wraps(buf, len, i + (size_t)limit, + (unsigned char)buf[i + limit - 1]); + if (!wrapped) break; + last_char = (unsigned char)buf[i + limit - 1]; + EMIT_STREAM_ROW_FAST(buf + i, limit, true); + i += (size_t)limit; + continue; + } + + int p1 = term_ctz64(mask); + if (p1 > cols) + break; + + int eol1 = 0; + if (buf[i + p1] == '\n' && lnm) { + eol1 = 1; + } else if (buf[i + p1] == '\r' && (p1 + 1 < 64 ? buf[i + p1 + 1] == '\n' : (i + p1 + 1 < len && buf[i + p1 + 1] == '\n'))) { + eol1 = 2; + } else if (p1 < cols) { + goto done; + } else { + break; + } + + if (p1 > 0) + last_char = (unsigned char)buf[i + p1 - 1]; + EMIT_STREAM_ROW_FAST(buf + i, p1, false); + + int next_start = p1 + eol1; + int last_consumed = next_start; + + if (next_start < 64) { + uint64_t mask2 = mask >> next_start; + if (mask2 != 0) { + int rel_p2 = term_ctz64(mask2); + int p2 = next_start + rel_p2; + if (rel_p2 <= cols) { + int eol2 = 0; + if (buf[i + p2] == '\n' && lnm) { + eol2 = 1; + } else if (buf[i + p2] == '\r' && (p2 + 1 < 64 ? buf[i + p2 + 1] == '\n' : (i + p2 + 1 < len && buf[i + p2 + 1] == '\n'))) { + eol2 = 2; + } + + if (eol2 > 0) { + if (rel_p2 > 0) + last_char = (unsigned char)buf[i + p2 - 1]; + EMIT_STREAM_ROW_FAST(buf + i + next_start, rel_p2, false); + + int next_start2 = p2 + eol2; + last_consumed = next_start2; + + if (next_start2 < 64) { + uint64_t mask3 = mask >> next_start2; + if (mask3 != 0) { + int rel_p3 = term_ctz64(mask3); + int p3 = next_start2 + rel_p3; + if (rel_p3 <= cols) { + int eol3 = 0; + if (buf[i + p3] == '\n' && lnm) { + eol3 = 1; + } else if (buf[i + p3] == '\r' && (p3 + 1 < 64 ? buf[i + p3 + 1] == '\n' : (i + p3 + 1 < len && buf[i + p3 + 1] == '\n'))) { + eol3 = 2; + } + + if (eol3 > 0) { + if (rel_p3 > 0) + last_char = (unsigned char)buf[i + p3 - 1]; + EMIT_STREAM_ROW_FAST(buf + i + next_start2, rel_p3, false); + + last_consumed = p3 + eol3; + } + } + } + } + } + } + } + } + + i += (size_t)last_consumed; + } + if (i >= len) break; + } +#endif + int limit = len - i > (size_t)cols ? cols : (int)(len - i); + int run = scan_ascii_run(buf + i, limit); + int eol = 0; + if (TERM_EXPECT(i + run + 1 < len && buf[i + run] == '\r' && buf[i + run + 1] == '\n', 1)) { + eol = 2; + } else if (i + run < len && buf[i + run] == '\n' && lnm) { + eol = 1; + } + + if (TERM_EXPECT(eol > 0, 1)) { + if (run > 0) + last_char = (unsigned char)buf[i + run - 1]; + EMIT_STREAM_ROW(buf + i, run, false); + i += (size_t)(run + eol); + continue; + } + + if (!run) { + int neol = newline_bytes(buf + i, len - i); + if (neol == 2 || (neol == 1 && lnm)) { + EMIT_STREAM_ROW(buf + i, 0, false); + i += (size_t)neol; + continue; + } + break; + } + + bool wrapped = run == cols && autowrap && + next_byte_wraps(buf, len, i + (size_t)run, + (unsigned char)buf[i + run - 1]); + if (!wrapped) break; + int full = cols + (int)scan_ascii_run(buf + i + cols, + (int)(len - i) - cols); + int bulk_lines = full / cols - (full % cols == 0); + if (bulk_lines > 0 && + (i + (size_t)full == len || + ((unsigned char)buf[i + full] >= 0x20 && + (unsigned char)buf[i + full] < 0x7f) || + full % cols == 0)) { + emit_stream_bulk_wrapped(t, s, buf + i, bulk_lines, cols, &attr, def_pen, + q, q_cap, &q_count, &orig_y, &orig_rem, + &flushed_scrolled); + size_t write_cells = (size_t)bulk_lines * (size_t)cols; + last_char = (unsigned char)buf[i + write_cells - 1]; + i += write_cells; + continue; + } + last_char = (unsigned char)buf[i + run - 1]; + EMIT_STREAM_ROW(buf + i, run, true); + i += (size_t)run; + } +#undef EMIT_STREAM_ROW +#undef EMIT_STREAM_ROW_FAST +#if defined(TERM_KERNEL_AVX512) +done: +#endif + if (i == 0) + return 0; + + size_t queued_rows = 0; + for (int j = 0; j < q_count; j++) + queued_rows += q[j].wrapped ? 1 : rows_narrow(q[j].len, cols); + size_t loop_scrolled = flushed_scrolled + queued_rows; + if (loop_scrolled > 0) { + int push_orig = queued_rows < (size_t)orig_rem ? (int)queued_rows : orig_rem; + if (push_orig > 0) + flush_orig_rows(t, s, &orig_y, &orig_rem, push_orig); + + int needed = rows - 1 - orig_rem; + int tail_rows = 0; + int k = q_count; + while (k > 0 && tail_rows < needed) { + k--; + struct hist_ascii_line pr = q[k]; + int r = (pr.wrapped || pr.len == 0) ? 1 : (int)rows_narrow(pr.len, cols); + tail_rows += r; + } + + int flush_n = k; + if (flush_n > 0) { + term_hist_push_ascii_lines(&t->hist, q, flush_n, &attr); + } + + int skip_rows = tail_rows - needed; + if (skip_rows > 0) { + struct hist_ascii_line pr = q[k]; + int skip_bytes = skip_rows * cols; + if (skip_bytes > (int)pr.len) skip_bytes = (int)pr.len; + term_hist_push_ascii(&t->hist, pr.ptr, skip_bytes, &attr, true); + } + + if (orig_rem == 0) { + s->top = 0; + } else { + s->top += tail_rows; + if (s->top >= rows) + s->top -= rows; + } + /* Rows that stay on the screen become text rows. A row that + * continues the row above starts from its pen. */ + int cur_y = orig_rem; + bool prev_wrapped = false; + for (int j = k; j < q_count && cur_y < rows - 1; j++) { + struct hist_ascii_line pr = q[j]; + int p_len = pr.len; + const char *p_ptr = pr.ptr; + + if (p_len == 0) { + int p = phys_row(t, cur_y); + struct srow *dst = &s->slot[p]; + dst->uid = ++t->next_row_uid; + dst->version++; + uint8_t tail_padded = (dst->kind == ROW_CELLS && dst->len == 0) + ? (dst->flags & RF_TAIL_PADDED) : 0; + dst->kind = ROW_CELLS; + dst->len = 0; + dst->flags = tail_padded; + prev_wrapped = false; + cur_y++; + continue; + } + + int off_in_line = 0; + if (j == k && skip_rows > 0) { + off_in_line = skip_rows * cols; + prev_wrapped = true; + } + + while (off_in_line < p_len && cur_y < rows - 1) { + int chunk = p_len - off_in_line; + bool row_wraps = false; + if (chunk > cols) { + chunk = cols; + row_wraps = true; + } else if (pr.wrapped && off_in_line + chunk == p_len) { + row_wraps = true; + } + + int p = phys_row(t, cur_y); + struct srow *dst = &s->slot[p]; + dst->uid = ++t->next_row_uid; + dst->version++; + + struct pen_attr start = prev_wrapped ? attr : pen_attr_default(); + dst->kind = ROW_TEXT; + dst->len = 0; + dst->flags = row_wraps ? RF_WRAPPED : 0; + text_row_start(&dst->text, &start); + dst->text.text = dst->ascii; + text_append(dst, (const uint8_t *)(p_ptr + off_in_line), + chunk, chunk, chunk, HLINE_NARROW, + attr.fg, attr.bg, pen_sl(&attr)); + + prev_wrapped = row_wraps; + off_in_line += chunk; + cur_y++; + } + } + while (cur_y < rows - 1) { + int p = phys_row(t, cur_y); + struct srow *dst = &s->slot[p]; + dst->uid = ++t->next_row_uid; + dst->version++; + dst->kind = ROW_CELLS; + dst->len = 0; + dst->flags = 0; + cur_y++; + } + int bot_p = phys_row(t, rows - 1); + struct srow *bot = &s->slot[bot_p]; + uint8_t bot_tail = (bot->kind == ROW_CELLS && bot->len == 0) + ? (bot->flags & RF_TAIL_PADDED) : 0; + bot->uid = ++t->next_row_uid; + bot->version++; + bot->kind = ROW_CELLS; + bot->len = 0; + bot->flags = bot_tail; + if (t->prompt_y > 0) { + t->prompt_y = (size_t)t->prompt_y > loop_scrolled + ? t->prompt_y - (int)loop_scrolled : 0; + } + clamp_selection_after_scroll(t); + } + + t->last_char = (uint32_t)last_char; + t->cursor_x = 0; + t->cursor_y = rows - 1; + t->pending_wrap = false; + t->full_gen++; + return i; +} + +#if defined(TERM_KERNEL_AVX512) +TERM_TARGET_AVX512 +static TERM_NOINLINE size_t bulk_ascii_lines_avx512(struct term *t, const char *buf, size_t len) +{ + return bulk_ascii_lines_body(t, buf, len, TERM_ISA_AVX512); +} +#endif + +static TERM_NOINLINE size_t bulk_ascii_lines_base(struct term *t, const char *buf, size_t len) +{ + return bulk_ascii_lines_body(t, buf, len, TERM_ISA_BASE); +} + +size_t term_sc_bulk_ascii_lines(struct term *t, const char *buf, size_t len) +{ + if (TERM_EXPECT(t->scr.rows > 256, 0)) return 0; + if (len >= 1024 && t->autowrap && using_default_colors(t) && !term_flags_pad(t)) { + size_t done = 0; + if (!term_lnm(t)) done = stream_ascii_wrapped(t, buf, len); + else if (!t->pending_wrap) done = stream_ascii_lf(t, buf, len); + if (done) return done; + } +#if defined(TERM_KERNEL_AVX512) + if (term_use_avx512()) + return bulk_ascii_lines_avx512(t, buf, len); +#endif + return bulk_ascii_lines_base(t, buf, len); +} diff --git a/src/core.h b/src/core.h new file mode 100644 index 0000000..2460938 --- /dev/null +++ b/src/core.h @@ -0,0 +1,385 @@ +#ifndef WSAP_TERM_CORE_H +#define WSAP_TERM_CORE_H + +#include "compat.h" +#include "internal.h" +#include "screen.h" + +#include <stdbool.h> +#include <stddef.h> +#include <stdint.h> + +struct span { + struct srow *row; + struct cell *cells; + int avail; + int x; + int old_len; +}; + +/* Begin writing a span of cells. Settle cursor row, repair left edge. */ +static inline struct span sc_span_begin(struct term *t, int want); + +/* Finish writing a span of cells. Repair right edge, raise len, bump gen. */ +static inline void sc_span_end(struct term *t, struct span *s, int cols, + uint8_t row_flags, uint32_t last); + +/* Print ASCII characters through span and SIMD chunking. */ +static inline void sc_print_ascii(struct term *t, const char *s, int n); + +/* Print one glyph (single codepoint or cluster handle). */ +void term_sc_print_glyph(struct term *t, uint32_t cp, int width, uint16_t flags); + +/* Extend the last printed glyph into a grapheme cluster. */ +bool term_sc_extend_cluster(struct term *t, uint32_t cp, int klass); + +/* Multi-line or region-constrained line feed helper. */ +void term_sc_linefeed_slow(struct term *t, size_t n); + +/* Line feed: move down or scroll with BCE. */ +static inline void sc_linefeed(struct term *t, size_t n) +{ + if (TERM_EXPECT(n == 1 && t->scroll_top == 0 && + t->scroll_bottom == t->rows - 1, 1)) { + t->pending_wrap = false; + term_row(t, t->cursor_y)->version++; + if (TERM_EXPECT(t->cursor_y == t->rows - 1, 1)) + term_screen_scroll_up(t); + else + t->cursor_y++; + return; + } + term_sc_linefeed_slow(t, n); +} + +/* Carriage return: move cursor to left margin. */ +static inline void sc_cr(struct term *t) +{ + t->pending_wrap = false; + if (t->cursor_x != 0) { + t->cursor_x = 0; + term_row(t, t->cursor_y)->version++; + } +} + +/* Move cursor to (x, y), margin-aware. */ +static inline void sc_move(struct term *t, int x, int y) +{ + t->pending_wrap = false; + if (x < 0) x = 0; + if (x >= t->cols) x = t->cols - 1; + if (y < 0) y = 0; + if (y >= t->rows) y = t->rows - 1; + + int old_y = t->cursor_y; + if (x == t->cursor_x && y == old_y) + return; + t->cursor_x = x; + t->cursor_y = y; + term_row(t, old_y)->version++; + if (y != old_y) + term_row(t, y)->version++; +} + +/* Advance cursor by n tab stops. */ +void term_sc_tab(struct term *t, int n); + +/* Move cursor back one column. */ +static inline void sc_bs(struct term *t) +{ + t->pending_wrap = false; + if (t->cursor_x > 0) + t->cursor_x--; + term_row(t, t->cursor_y)->version++; +} + +/* Erase cells in span [from, to) on physical row y. */ +static inline void sc_erase_span(struct term *t, int y, int from, int to) +{ + t->pending_wrap = false; + if (to > t->cols) to = t->cols; + int p = phys_row(t, y); + erase_span(t, p, from, to); + t->scr.slot[p].version++; +} + +/* Erase display mode 0, 1, 2, or 3. */ +static inline void sc_erase_display(struct term *t, int mode) +{ + t->pending_wrap = false; + term_screen_erase(t, mode); +} + +/* Erase line mode 0, 1, or 2. */ +static inline void sc_erase_line(struct term *t, int mode) +{ + t->pending_wrap = false; + term_screen_erase_line(t, mode); +} + +/* Insert n blank characters at cursor. */ +static inline void sc_insert_chars(struct term *t, int n) +{ + t->pending_wrap = false; + term_screen_insert_chars(t, n); +} + +/* Delete n characters at cursor. */ +static inline void sc_delete_chars(struct term *t, int n) +{ + t->pending_wrap = false; + term_screen_delete_chars(t, n); +} + +/* Scroll margins: top to bottom, n lines up or down. */ +void term_sc_scroll_region(struct term *t, int top, int bottom, int n, bool down); + +/* Scroll the active scroll region up or down by n lines (with history). */ +void term_sc_scroll(struct term *t, int n, bool down); + +/* Restore saved cursor state and position. */ +static inline void sc_restore_cursor(struct term *t, const struct term_saved_cursor *s) +{ + term_cursor_apply(t, s); + sc_move(t, s->x, s->y); + t->pending_wrap = s->pending_wrap; +} + +/* Change attributes in a rectangular or stream area (DECCARA). */ +void term_sc_deccara(struct term *t, const struct csi *csi); + +/* Screen alignment pattern (DECALN). */ +void term_sc_decaln(struct term *t); + +/* Reset screen and cursor state for full reset (RIS). */ +void term_sc_reset(struct term *t); + +/* Set reverse video mode (DECSCNM). */ +void term_sc_set_reverse_video(struct term *t, bool enabled); + +/* Set autowrap mode (DECAWM). */ +void term_sc_set_autowrap(struct term *t, bool enabled); + +/* Set cursor style (DECSCUSR). */ +void term_sc_set_cursor_style(struct term *t, int ps); + +/* Bulk streaming of ASCII lines with raw rows and skip-ahead. */ +size_t term_sc_bulk_ascii_lines(struct term *t, const char *buf, size_t len); + +/* Fast CSI parser. */ +int term_try_fast_csi(struct term *t, const char *buf, size_t rem); + +static inline struct span sc_span_begin(struct term *t, int want) +{ + (void)want; + if (TERM_EXPECT(t->pending_wrap, 0)) { + row_set_wrapped(term_row(t, t->cursor_y), true); + t->cursor_x = t->lr_margin_mode ? t->left_margin : 0; + sc_linefeed(t, 1); + t->pending_wrap = false; + } + struct srow *row = term_row(t, t->cursor_y); + if (TERM_EXPECT(row->kind != ROW_CELLS, 0)) + srow_settle(row, t->cols); + int cx = t->cursor_x; + term_row_fill_gap(row, cx); + int len = row->len; + if (TERM_EXPECT(cx < len && (row->cells[cx].flags & CELL_TRAIL), 0)) + fill_cells(row->cells + cx - 1, 1, row->cells[cx - 1].fg, row->cells[cx - 1].bg); + + int right = TERM_EXPECT(t->lr_margin_mode, 0) ? t->right_margin : t->cols - 1; + int avail = right + 1 > cx ? right + 1 - cx : 0; + return (struct span){ + .row = row, + .cells = row->cells + cx, + .avail = avail, + .x = cx, + .old_len = len, + }; +} + +static inline void sc_span_end(struct term *t, struct span *s, int cols, + uint8_t row_flags, uint32_t last) +{ + struct srow *row = s->row; + int end_x = s->x + cols; + if (TERM_EXPECT(end_x < s->old_len && (row->cells[end_x].flags & CELL_TRAIL), 0)) + fill_cells(row->cells + end_x, 1, row->cells[end_x].fg, row->cells[end_x].bg); + if (end_x > s->old_len) + row->len = (uint16_t)end_x; + if (TERM_EXPECT(row_flags != 0, 0)) + row->flags |= row_flags; + row->version++; + t->last_char = last; + + if (TERM_EXPECT(cols < s->avail, 1)) { + t->cursor_x = end_x; + t->pending_wrap = false; + } else { + t->cursor_x = s->x + s->avail - 1; + t->pending_wrap = t->autowrap; + } +} + +/* Move the cursor after cols columns written at x, of avail. */ +static inline void sc_text_advance(struct term *t, int x, int cols, int avail) +{ + if (cols < avail) { + t->cursor_x = x + cols; + t->pending_wrap = false; + } else { + t->cursor_x = x + avail - 1; + t->pending_wrap = t->autowrap; + } +} + +/* Append ASCII at the cursor, with the pen fg, bg, sl (flags | link << 16), + * when the cursor row takes text. Return the bytes taken. */ +static inline int sc_text_ascii(struct term *t, const char *s, int n, uint32_t fg, uint32_t bg, + uint32_t sl) +{ + if (TERM_EXPECT(t->insert_mode || t->lr_margin_mode, 0)) + return 0; + if (t->pending_wrap) { + row_set_wrapped(term_row(t, t->cursor_y), true); + t->cursor_x = 0; + sc_linefeed(t, 1); + t->pending_wrap = false; + } + struct srow *row = text_append_row(t); + if (!row) + return 0; + int x = t->cursor_x, avail = t->cols - x; + int take = n < avail ? n : avail; + if (take <= 0) + return 0; + text_append(row, (const uint8_t *)s, take, take, take, HLINE_NARROW, fg, bg, sl); + t->last_char = (unsigned char)s[take - 1]; + sc_text_advance(t, x, take, avail); + return take; +} + +static inline void sc_print_ascii(struct term *t, const char *s, int n) +{ + uint32_t fg = TERM_EXPECT(t->inverse, 0) ? t->cur_bg : t->cur_fg; + uint32_t bg = TERM_EXPECT(t->inverse, 0) ? t->cur_fg : t->cur_bg; + uint32_t flags = term_flags_pad(t); + + for (int k; n > 0 && (k = sc_text_ascii(t, s, n, fg, bg, flags)) > 0; s += k, n -= k) {} + + while (n > 0) { + struct span span = sc_span_begin(t, n); + int take = n < span.avail ? n : span.avail; + if (TERM_EXPECT(take <= 0, 0)) { + t->pending_wrap = t->autowrap; + if (!t->autowrap) + break; + span = sc_span_begin(t, n); + take = n < span.avail ? n : span.avail; + if (take <= 0) + break; + } + put_ascii_chunk(span.cells, s, take, fg, bg, flags); + sc_span_end(t, &span, take, 0, (unsigned char)s[take - 1]); + if (TERM_EXPECT(n <= take, 1)) break; + s += take; + n -= take; + } +} + +/* Write a short ASCII span without settling an already writable row. */ +static inline bool sc_ascii_fast(struct term *t, const char *s, int n, + uint32_t fg, uint32_t bg, uint32_t flags_pad) +{ + int cx = t->cursor_x; + if (TERM_EXPECT(t->pending_wrap || t->insert_mode || + t->lr_margin_mode || t->inverse || + n <= 0 || n > t->cols - cx, 0)) + return false; + struct srow *row = term_row(t, t->cursor_y); + if (row->kind == ROW_TEXT || (row->len == 0 && cx == 0)) { + if (sc_text_ascii(t, s, n, fg, bg, flags_pad) == n) + return true; + if (row->kind == ROW_TEXT) + return false; + } + if (TERM_EXPECT(row->kind != ROW_CELLS || cx > row->len, 0)) + return false; + int len = row->len; + struct cell *cells = row->cells; + if (TERM_EXPECT((cx < len && (cells[cx].flags & CELL_TRAIL)) || + (cx + n < len && (cells[cx + n].flags & CELL_TRAIL)), 0)) + return false; + put_ascii_chunk(cells + cx, s, n, fg, bg, flags_pad); + cx += n; + if (cx > len) + row->len = (uint16_t)cx; + row->version++; + t->last_char = (unsigned char)s[n - 1]; + if (cx < t->cols) { + t->cursor_x = cx; + t->pending_wrap = false; + } else { + t->cursor_x = t->cols - 1; + t->pending_wrap = t->autowrap; + } + return true; +} + +/* Fast inline check for 1-character write (fg-char fast path). */ +static inline bool sc_put1_fast(struct term *t, char ch, uint32_t fg, uint32_t bg, + uint32_t flags_pad) +{ + int cx = t->cursor_x; + int cols = t->cols; + if (TERM_EXPECT(t->pending_wrap | t->insert_mode | t->lr_margin_mode | t->inverse, 0)) + return false; + if (TERM_EXPECT(cx >= cols, 0)) + return false; + struct srow *row = term_row(t, t->cursor_y); + if (row->kind == ROW_TEXT) { + struct text_row *tr = &row->text; + if (TERM_EXPECT(cx != tr->cells || !tr->cells || cx + 1 == cols, 0)) + return sc_text_ascii(t, &ch, 1, fg, bg, flags_pad) == 1; + /* One character in the middle of a text row. */ + tr->text[tr->text_len++] = (uint8_t)ch; + tr->cells++; + hist_runs_add_pen(&tr->runs, fg, bg, flags_pad, 1); + if (flags_pad >> 16) + tr->flags |= HLINE_HAS_LINK; + if (tr->cells > row->len) + row->len = tr->cells; + row->version++; + t->last_char = (uint8_t)ch; + t->cursor_x = cx + 1; + return true; + } + if (row->len == 0 && cx == 0) { + if (sc_text_ascii(t, &ch, 1, fg, bg, flags_pad) == 1) + return true; + } + int len = row->len; + if (TERM_EXPECT(cx > len, 0)) + return false; + struct cell *cells = row->cells; + if (TERM_EXPECT((cx < len && (cells[cx].flags & CELL_TRAIL)) || + (cx + 1 < len && (cells[cx + 1].flags & CELL_TRAIL)), 0)) + return false; + + cells[cx] = (struct cell){ (uint8_t)ch, fg, bg, + (uint16_t)flags_pad, (uint16_t)(flags_pad >> 16) }; + t->last_char = (uint8_t)ch; + cx++; + if (cx > len) + row->len = (uint16_t)cx; + row->version++; + if (TERM_EXPECT(cx < cols, 1)) { + t->cursor_x = cx; + } else { + t->cursor_x = cols - 1; + t->pending_wrap = t->autowrap; + } + return true; +} + +#endif diff --git a/src/csi.h b/src/csi.h new file mode 100644 index 0000000..b4609ea --- /dev/null +++ b/src/csi.h @@ -0,0 +1,109 @@ +#ifndef CSI_H +#define CSI_H + +#include <stddef.h> +#include <stdint.h> +#include <limits.h> + +#define MAX_PARAMS 16 + +struct csi { + int params[MAX_PARAMS]; + int nparams; + uint16_t subparams; + char prefix, intermediate, final; +}; + +enum csi_phase { CSI_ENTRY, CSI_PARAMETERS, CSI_INTERMEDIATES, CSI_IGNORE }; +enum csi_result { CSI_MORE, CSI_DISPATCH, CSI_DISCARD, CSI_CONTROL }; + +struct csi_parser { + struct csi csi; + enum csi_phase phase; +}; + +static inline void csi_parser_reset(struct csi_parser *p) +{ + p->csi.params[0] = 0; + p->csi.nparams = 0; + p->csi.subparams = 0; + p->csi.prefix = p->csi.intermediate = p->csi.final = 0; + p->phase = CSI_ENTRY; +} + +static inline int csi_count(const struct csi *csi) +{ + return csi->params[0] > 0 ? csi->params[0] : 1; +} + +static inline enum csi_result csi_read(struct csi_parser *restrict p, const char *restrict s, + size_t len, size_t *restrict consumed) +{ + size_t i = 0; + enum csi_result result = CSI_MORE; + while (i < len) { + unsigned char ch = (unsigned char)s[i]; + if (ch < 0x20 || ch >= 0x80) { + result = CSI_CONTROL; + break; + } + i++; + if (ch == 0x7f) continue; + if (ch >= 0x40) { + p->csi.final = (char)ch; + result = p->phase == CSI_IGNORE ? CSI_DISCARD : CSI_DISPATCH; + break; + } + if (p->phase == CSI_IGNORE) continue; + + if (ch >= '0' && ch <= ';') { + if (p->phase == CSI_INTERMEDIATES) { + p->phase = CSI_IGNORE; + continue; + } + p->phase = CSI_PARAMETERS; + int n = p->csi.nparams ? p->csi.nparams : 1; + int value = p->csi.params[n - 1]; + for (;;) { + if (ch <= '9') { + for (;;) { + int digit = ch - '0'; + value = value < 214748364 ? value * 10 + digit : INT_MAX; + if (i == len) break; + ch = (unsigned char)s[i]; + if (ch < '0' || ch > '9') break; + i++; + } + } else { + p->csi.params[n - 1] = value; + if (n == MAX_PARAMS) { + p->phase = CSI_IGNORE; + break; + } + if (ch == ':') p->csi.subparams |= (uint16_t)(1u << n); + n++; + value = 0; + } + if (i == len) break; + ch = (unsigned char)s[i]; + if (ch < '0' || ch > ';') break; + i++; + } + p->csi.params[n - 1] = value; + p->csi.nparams = n; + } else if (ch >= '<') { + if (p->phase != CSI_ENTRY || p->csi.prefix) p->phase = CSI_IGNORE; + else p->csi.prefix = (char)ch; + } else { + if (p->phase == CSI_INTERMEDIATES) p->phase = CSI_IGNORE; + else { + p->phase = CSI_INTERMEDIATES; + p->csi.intermediate = (char)ch; + } + } + } + *consumed = i; + return result; +} + +#endif diff --git a/src/escape.c b/src/escape.c new file mode 100644 index 0000000..119010c --- /dev/null +++ b/src/escape.c @@ -0,0 +1,1116 @@ +#include "escape.h" +#include "compat.h" +#include "core.h" +#include "kernels.h" +#include "pen.h" +#include "screen.h" +#include "text.h" +#include "osc/osc.h" +#include <stdarg.h> +#include <stdlib.h> +#include <string.h> +#include <stdio.h> + +static int sgr_color_param(const struct term *t, char *buf, size_t sz, int base, uint32_t c) +{ + if (term_color_is_slot(c) && (c & 0x1FFu) < 256) { + int i = (int)(c & 0xFFu); + /* The underline color has no short form. */ + if (i < 16 && base != 50) + return snprintf(buf, sz, ";%d", i < 8 ? base + i : base + 60 + i - 8); + return snprintf(buf, sz, ";%d:5:%d", base + 8, i); + } + c = palette_resolve(&t->palette, c); + return snprintf(buf, sz, ";%d:2:%d:%d:%d", base + 8, + (int)((c >> 16) & 0xff), (int)((c >> 8) & 0xff), (int)(c & 0xff)); +} + +static int sgr_serialize(const struct term *t, char *buf, size_t sz) +{ + int n = 0; + if (!(t->bold || t->faint || t->italic || t->underline || t->strike || + t->overline || t->inverse || t->cur_fg != TERM_COLOR_FG || + t->cur_bg != TERM_COLOR_BG || t->cur_ul)) + return 0; +#define SGR_ADD(expr) \ + do { \ + n += (expr); \ + if ((size_t)n >= sz) return sz ? (int)sz - 1 : 0; \ + } while (0) + SGR_ADD(snprintf(buf + n, sz - (size_t)n, "0")); + if (t->bold) SGR_ADD(snprintf(buf + n, sz - (size_t)n, ";1")); + if (t->faint) SGR_ADD(snprintf(buf + n, sz - (size_t)n, ";2")); + if (t->italic) SGR_ADD(snprintf(buf + n, sz - (size_t)n, ";3")); + if (t->underline == 1) SGR_ADD(snprintf(buf + n, sz - (size_t)n, ";4")); + else if (t->underline) SGR_ADD(snprintf(buf + n, sz - (size_t)n, ";4:%d", t->underline)); + if (t->inverse) SGR_ADD(snprintf(buf + n, sz - (size_t)n, ";7")); + if (t->strike) SGR_ADD(snprintf(buf + n, sz - (size_t)n, ";9")); + if (t->overline) SGR_ADD(snprintf(buf + n, sz - (size_t)n, ";53")); + if (t->cur_fg != TERM_COLOR_FG) SGR_ADD(sgr_color_param(t, buf + n, sz - (size_t)n, 30, t->cur_fg)); + if (t->cur_bg != TERM_COLOR_BG) SGR_ADD(sgr_color_param(t, buf + n, sz - (size_t)n, 40, t->cur_bg)); + if (t->cur_ul) SGR_ADD(sgr_color_param(t, buf + n, sz - (size_t)n, 50, t->cur_ul)); +#undef SGR_ADD + return n; +} + +static int dcs_decode_hex(const char *s, int len, char *out, int cap) +{ + if (len & 1) return -1; + int n = 0; + for (int i = 0; i < len; i += 2) { + int hi = term_hex_nib(s[i]), lo = term_hex_nib(s[i + 1]); + if (hi < 0 || lo < 0 || n >= cap) return -1; + out[n++] = (char)(hi << 4 | lo); + } + return n; +} + +static const char *term_tcap(const struct term *t, const char *name) +{ + static const struct { const char *name, *value; } caps[] = { + { "Co", "16777216" }, + { "RGB", "8/8/8" }, + { "colors", "16777216" }, + { "kDC", "\x1b[3;2~" }, { "kDC3", "\x1b[3;3~" }, + { "kDC4", "\x1b[3;4~" }, { "kDC5", "\x1b[3;5~" }, + { "kDC6", "\x1b[3;6~" }, { "kDC7", "\x1b[3;7~" }, + { "kDN", "\x1b[1;2B" }, { "kDN3", "\x1b[1;3B" }, + { "kDN4", "\x1b[1;4B" }, { "kDN5", "\x1b[1;5B" }, + { "kDN6", "\x1b[1;6B" }, { "kDN7", "\x1b[1;7B" }, + { "kEND", "\x1b[1;2F" }, { "kEND3", "\x1b[1;3F" }, + { "kEND4", "\x1b[1;4F" }, { "kEND5", "\x1b[1;5F" }, + { "kEND6", "\x1b[1;6F" }, { "kEND7", "\x1b[1;7F" }, + { "kHOM", "\x1b[1;2H" }, { "kHOM3", "\x1b[1;3H" }, + { "kHOM4", "\x1b[1;4H" }, { "kHOM5", "\x1b[1;5H" }, + { "kHOM6", "\x1b[1;6H" }, { "kHOM7", "\x1b[1;7H" }, + { "kIC", "\x1b[2;2~" }, { "kIC3", "\x1b[2;3~" }, + { "kIC4", "\x1b[2;4~" }, { "kIC5", "\x1b[2;5~" }, + { "kIC6", "\x1b[2;6~" }, { "kIC7", "\x1b[2;7~" }, + { "kLFT", "\x1b[1;2D" }, { "kLFT3", "\x1b[1;3D" }, + { "kLFT4", "\x1b[1;4D" }, { "kLFT5", "\x1b[1;5D" }, + { "kLFT6", "\x1b[1;6D" }, { "kLFT7", "\x1b[1;7D" }, + { "kNXT", "\x1b[6;2~" }, { "kNXT3", "\x1b[6;3~" }, + { "kNXT4", "\x1b[6;4~" }, { "kNXT5", "\x1b[6;5~" }, + { "kNXT6", "\x1b[6;6~" }, { "kNXT7", "\x1b[6;7~" }, + { "kPRV", "\x1b[5;2~" }, { "kPRV3", "\x1b[5;3~" }, + { "kPRV4", "\x1b[5;4~" }, { "kPRV5", "\x1b[5;5~" }, + { "kPRV6", "\x1b[5;6~" }, { "kPRV7", "\x1b[5;7~" }, + { "kRIT", "\x1b[1;2C" }, { "kRIT3", "\x1b[1;3C" }, + { "kRIT4", "\x1b[1;4C" }, { "kRIT5", "\x1b[1;5C" }, + { "kRIT6", "\x1b[1;6C" }, { "kRIT7", "\x1b[1;7C" }, + { "kUP", "\x1b[1;2A" }, { "kUP3", "\x1b[1;3A" }, + { "kUP4", "\x1b[1;4A" }, { "kUP5", "\x1b[1;5A" }, + { "kUP6", "\x1b[1;6A" }, { "kUP7", "\x1b[1;7A" }, + { "kbs", "\x7f" }, { "kcbt", "\x1b[Z" }, + { "kcub1", "\x1bOD" }, { "kcud1", "\x1bOB" }, + { "kcuf1", "\x1bOC" }, { "kcuu1", "\x1bOA" }, + { "kdch1", "\x1b[3~" }, { "kend", "\x1bOF" }, + { "kent", "\x1bOM" }, { "khome", "\x1bOH" }, + { "kich1", "\x1b[2~" }, { "kind", "\x1b[1;2B" }, + { "kmous", "\x1b[<" }, { "knp", "\x1b[6~" }, + { "kpp", "\x1b[5~" }, { "kri", "\x1b[1;2A" }, + { "kxIN", "\x1b[I" }, { "kxOUT", "\x1b[O" }, + { "kf1", "\x1bOP" }, { "kf2", "\x1bOQ" }, + { "kf3", "\x1bOR" }, { "kf4", "\x1bOS" }, + { "kf5", "\x1b[15~" }, { "kf6", "\x1b[17~" }, + { "kf7", "\x1b[18~" }, { "kf8", "\x1b[19~" }, + { "kf9", "\x1b[20~" }, { "kf10", "\x1b[21~" }, + { "kf11", "\x1b[23~" }, { "kf12", "\x1b[24~" }, + { "kf13", "\x1b[1;2P" }, { "kf14", "\x1b[1;2Q" }, + { "kf15", "\x1b[1;2R" }, { "kf16", "\x1b[1;2S" }, + { "kf17", "\x1b[15;2~" }, { "kf18", "\x1b[17;2~" }, + { "kf19", "\x1b[18;2~" }, { "kf20", "\x1b[19;2~" }, + { "kf21", "\x1b[20;2~" }, { "kf22", "\x1b[21;2~" }, + { "kf23", "\x1b[23;2~" }, { "kf24", "\x1b[24;2~" }, + { "kf25", "\x1b[1;5P" }, { "kf26", "\x1b[1;5Q" }, + { "kf27", "\x1b[1;5R" }, { "kf28", "\x1b[1;5S" }, + { "kf29", "\x1b[15;5~" }, { "kf30", "\x1b[17;5~" }, + { "kf31", "\x1b[18;5~" }, { "kf32", "\x1b[19;5~" }, + { "kf33", "\x1b[20;5~" }, { "kf34", "\x1b[21;5~" }, + { "kf35", "\x1b[23;5~" }, { "kf36", "\x1b[24;5~" }, + { "kf37", "\x1b[1;6P" }, { "kf38", "\x1b[1;6Q" }, + { "kf39", "\x1b[1;6R" }, { "kf40", "\x1b[1;6S" }, + { "kf41", "\x1b[15;6~" }, { "kf42", "\x1b[17;6~" }, + { "kf43", "\x1b[18;6~" }, { "kf44", "\x1b[19;6~" }, + { "kf45", "\x1b[20;6~" }, { "kf46", "\x1b[21;6~" }, + { "kf47", "\x1b[23;6~" }, { "kf48", "\x1b[24;6~" }, + }; + if (!strcmp(name, "TN") || !strcmp(name, "name")) + return t->tcap_name; + for (size_t i = 0; i < sizeof(caps) / sizeof(caps[0]); i++) + if (!strcmp(name, caps[i].name)) + return caps[i].value; + return NULL; +} + +struct resp_buf { + char *s; + size_t len, cap; + bool failed; +}; + +static void resp_put(struct resp_buf *r, const char *s, size_t n) +{ + if (r->failed) + return; + if (r->len + n + 1 > r->cap) { + size_t cap = r->cap ? r->cap : 256; + while (cap < r->len + n + 1) + cap *= 2; + char *p = realloc(r->s, cap); + if (!p) { + r->failed = true; + return; + } + r->s = p; + r->cap = cap; + } + memcpy(r->s + r->len, s, n); + r->len += n; + r->s[r->len] = '\0'; +} + +static void dcs_xtgettcap(struct term *t, const char *buf, int len) +{ + static const char hex[] = "0123456789abcdef"; + struct resp_buf r = {0}; + resp_put(&r, "\033P0+r", 5); + char *name = malloc((size_t)len / 2 + 1); + if (!name) { + free(r.s); + return; + } + int i = 0; + bool first = true; + while (i < len) { + int span = 0; + while (i + span < len && buf[i + span] != ';') span++; + int n = dcs_decode_hex(buf + i, span, name, len / 2); + if (n < 0) + break; + name[n] = 0; + const char *cap = term_tcap(t, name); + if (first && r.s) { r.s[2] = cap ? '1' : '0'; first = false; } + resp_put(&r, buf + i, (size_t)span); + if (!cap) break; + resp_put(&r, "=", 1); + for (const char *c = cap; *c; c++) { + unsigned char b = (unsigned char)*c; + char pair[2] = { hex[b >> 4], hex[b & 15] }; + resp_put(&r, pair, 2); + } + i += span; + if (i < len) resp_put(&r, &buf[i++], 1); + } + free(name); + resp_put(&r, "\033\\", 2); + if (!r.failed) + send_response(t, r.s); + free(r.s); +} + +static int dcs_param_int(const char **p, const char *end) +{ + int v = 0, any = 0; + while (*p < end && **p >= '0' && **p <= '9') { + v = v < 214748364 ? v * 10 + (**p - '0') : INT_MAX; + (*p)++; + any = 1; + } + if (*p < end && **p == ';') (*p)++; + return any ? v : -1; +} + +static int dcs_param_chr(const char **p, const char *end) +{ + if (*p >= end) return -1; + int v = (unsigned char)*(*p)++; + if (*p < end && **p == ';') (*p)++; + return v; +} + +static void restore_deccir(struct term *t, const char *p, const char *end) +{ + int row = dcs_param_int(&p, end); + if (row <= 0 || row > t->rows) return; + int col = dcs_param_int(&p, end); + if (col <= 0 || col > t->cols) return; + if (dcs_param_int(&p, end) != 1) return; + sc_move(t, col - 1, row - 1); + int rend = dcs_param_chr(&p, end); + if ((rend & 0xf0) != 0x40) return; + t->bold = rend & 1; + t->underline = (rend & 2) ? 1 : 0; + t->inverse = (rend & 8) != 0; + int attr = dcs_param_chr(&p, end); + if ((attr & 0xf0) != 0x40) return; + int flag = dcs_param_chr(&p, end); + if ((flag & 0xf0) != 0x40) return; + t->autowrap = (flag & 8) != 0; + t->origin_mode = (flag & 1) != 0; + int curss = (flag & 4) ? 3 : (flag & 2) ? 2 : 0; + int gl = dcs_param_chr(&p, end) - '0'; + if (gl < 0 || gl > 3) return; + int gr = dcs_param_chr(&p, end) - '0'; + if (gr < 0 || gr > 3) return; + t->charset_active = gl < 2 ? gl : (curss ? 1 : 0); + if (dcs_param_chr(&p, end) < 0) return; + for (int i = 0; i < 4 && p < end; i++) { + char d = *p++; + if ((d == '%' || d == '&' || d == '"') && p < end) d = *p++; + if (i < 2) t->charsets[i] = d; + } + t->graphics = t->charsets[t->charset_active] == '0'; +} + +static void restore_dectabsr(struct term *t, const char *p, const char *end) +{ + term_tabs_clear(t); + int stop = 0, any = 0; + for (; p < end; p++) { + if (*p >= '0' && *p <= '9') { + stop = stop < 214748364 ? stop * 10 + (*p - '0') : INT_MAX; + any = 1; + } else if (*p == '/') { + if (any && --stop >= 0 && stop < t->tabs_cap) t->tabs[stop] = 1; + stop = any = 0; + } else { + return; + } + } + if (any && --stop >= 0 && stop < t->tabs_cap) t->tabs[stop] = 1; +} + +/* Grows the DCS string to hold extra more bytes, up to TERM_OSC_MAX. + * False when nothing more fits. */ +static bool dcs_reserve(struct term *t, size_t extra) +{ + size_t need = (size_t)t->dcs_len + extra; + if (need <= t->dcs_cap) + return true; + if (t->dcs_cap >= TERM_OSC_MAX) + return false; + size_t cap = t->dcs_cap ? t->dcs_cap : 256; + while (cap < need && cap < TERM_OSC_MAX) + cap *= 2; + if (cap > TERM_OSC_MAX) + cap = TERM_OSC_MAX; + char *p = realloc(t->dcs_buf, cap); + if (!p) + return false; + t->dcs_buf = p; + t->dcs_cap = cap; + return need <= cap; +} + +static void dcs_dispatch(struct term *t) +{ + if (term_str_was_truncated(t)) + return; + const char *buf = t->dcs_buf; + int len = t->dcs_len; + char resp[512]; + if (len < 2) return; + if (buf[0] == '+' && buf[1] == 'q') { + dcs_xtgettcap(t, buf + 2, len - 2); + } else if (buf[0] == '+' && buf[1] == 'p') { + int n = dcs_decode_hex(buf + 2, len - 2, t->tcap_name, sizeof(t->tcap_name) - 1); + if (n >= 0) t->tcap_name[n] = 0; + } else if (buf[0] == '+' && buf[1] == 'Q') { + int span = 0; + while (span < len - 2 && buf[2 + span] != ';') span++; + char *name = malloc((size_t)span / 2 + 1); + if (name && span && dcs_decode_hex(buf + 2, span, name, span / 2) >= 0) { + struct resp_buf r = {0}; + resp_put(&r, "\033P0+R", 5); + resp_put(&r, buf + 2, (size_t)span); + resp_put(&r, "\033\\", 2); + if (!r.failed) + send_response(t, r.s); + free(r.s); + } + free(name); + } else if (buf[0] == '$' && buf[1] == 'q') { + const char *spec = buf + 2; + int slen = len - 2, p; + if (slen == 2 && spec[0] == ' ' && spec[1] == 'q') { + int shape = t->cursor_style == TERM_CURSOR_BLOCK ? (t->cursor_blink ? 1 : 2) + : t->cursor_style == TERM_CURSOR_UNDERLINE ? (t->cursor_blink ? 3 : 4) + : (t->cursor_blink ? 5 : 6); + p = snprintf(resp, sizeof(resp), "\033P1$r%d q\033\\", shape); + } else if (slen == 1 && spec[0] == 'm') { + char sgr[256] = ""; + sgr_serialize(t, sgr, sizeof(sgr)); + p = snprintf(resp, sizeof(resp), "\033P1$r%sm\033\\", sgr); + } else if (slen == 1 && spec[0] == 'r') { + p = snprintf(resp, sizeof(resp), "\033P1$r%d;%dr\033\\", + t->scroll_top + 1, t->scroll_bottom + 1); + } else if (slen == 1 && spec[0] == 's') { + p = snprintf(resp, sizeof(resp), "\033P1$r%d;%ds\033\\", + t->left_margin + 1, t->right_margin + 1); + } else if (slen == 1 && spec[0] == 't') { + p = snprintf(resp, sizeof(resp), "\033P1$r%dt\033\\", t->rows); + } else if (slen == 2 && spec[0] == '"' && spec[1] == 'p') { + p = snprintf(resp, sizeof(resp), "\033P1$r64;1\"p\033\\"); + } else if (slen == 2 && spec[0] == '"' && spec[1] == 'q') { + p = snprintf(resp, sizeof(resp), "\033P1$r0\"q\033\\"); + } else if (slen == 2 && spec[0] == '$' && spec[1] == '|') { + p = snprintf(resp, sizeof(resp), "\033P1$r%d$|\033\\", t->cols); + } else if (slen == 2 && spec[0] == '$' && spec[1] == '}') { + p = snprintf(resp, sizeof(resp), "\033P1$r0$}\033\\"); + } else if (slen == 2 && spec[0] == '$' && spec[1] == '~') { + p = snprintf(resp, sizeof(resp), "\033P1$r0$~\033\\"); + } else if (slen == 2 && spec[0] == '*' && spec[1] == 'x') { + p = snprintf(resp, sizeof(resp), "\033P1$r%d*x\033\\", t->decsace); + } else if (slen == 2 && spec[0] == '*' && spec[1] == '|') { + p = snprintf(resp, sizeof(resp), "\033P1$r%d*|\033\\", t->rows); + } else { + p = snprintf(resp, sizeof(resp), "\033P0$r\033\\"); + } + (void)p; + send_response(t, resp); + } else if (buf[0] == '=' && len >= 3 && buf[len - 1] == 's') { + const char *p = buf + 1, *end = buf + len; + int mode = dcs_param_int(&p, end); + if (mode == 1) t->synchronized_output = true; + else if (mode == 2) t->synchronized_output = false; + } else if (buf[0] >= '0' && buf[0] <= '9') { + const char *p = buf, *end = buf + len; + int ps = dcs_param_int(&p, end); + while (p < end && *p == ';') { p++; dcs_param_int(&p, end); } + if (end - p >= 2 && p[0] == '$' && p[1] == 't') { + if (ps == 1) restore_deccir(t, p + 2, end); + else if (ps == 2) restore_dectabsr(t, p + 2, end); + } + } +} + +static void soft_reset(struct term *t) +{ + t->application_cursor = t->application_keypad = t->backarrow_sends_bs = false; + t->insert_mode = t->origin_mode = t->reverse_video = t->lnm = false; + t->focus_events = t->synchronized_output = t->lr_margin_mode = false; + t->mouse_mode = t->mouse_encoding = 0; + t->bracketed_paste = false; + t->autowrap = t->cursor_visible = true; + t->cursor_blink = t->default_cursor_blink; + t->cursor_style = t->default_cursor_style; + t->scroll_top = 0; t->scroll_bottom = t->rows - 1; + t->left_margin = 0; t->right_margin = t->cols - 1; + t->decsace = 0; + t->cur_link = 0; + memset(t->keyboard, 0, sizeof(t->keyboard)); + reset_sgr(t); +} + +static inline void sgr_set_slot(struct term *t, int code, uint32_t c) +{ + if (code == 38) t->cur_fg = c; + else if (code == 48) t->cur_bg = c; + else t->cur_ul = c; +} + +static inline void handle_sgr(struct term *t, const struct csi *csi) +{ + if (csi->nparams == 0) { + reset_sgr(t); + return; + } + + if (csi->nparams == 1 && !csi->subparams) { + int p = csi->params[0]; + if (p == 0) { reset_sgr(t); return; } + if ((unsigned)(p - 30) < 8) { t->cur_fg = term_color_index(p - 30); return; } + if ((unsigned)(p - 40) < 8) { t->cur_bg = term_color_index(p - 40); return; } + if ((unsigned)(p - 90) < 8) { t->cur_fg = term_color_index(p - 90 + 8); return; } + if ((unsigned)(p - 100) < 8) { t->cur_bg = term_color_index(p - 100 + 8); return; } + if (p == 1) { t->bold = true; return; } + if (p == 39) { t->cur_fg = TERM_COLOR_FG; return; } + if (p == 49) { t->cur_bg = TERM_COLOR_BG; return; } + } + + if (!csi->subparams) { + if (csi->nparams == 5 && csi->params[1] == 2 && + (csi->params[0] == 38 || csi->params[0] == 48)) { + sgr_set_slot(t, csi->params[0], + term_rgb(csi->params[2], csi->params[3], csi->params[4])); + return; + } + + if (csi->nparams == 10 && csi->params[1] == 2 && csi->params[6] == 2 && + (csi->params[0] == 38 || csi->params[0] == 48) && + csi->params[5] == (csi->params[0] == 38 ? 48 : 38)) { + sgr_set_slot(t, csi->params[0], + term_rgb(csi->params[2], csi->params[3], csi->params[4])); + sgr_set_slot(t, csi->params[5], + term_rgb(csi->params[7], csi->params[8], csi->params[9])); + return; + } + + if (csi->nparams == 3 && csi->params[1] == 5 && + (csi->params[0] == 38 || csi->params[0] == 48) && + (unsigned)csi->params[2] < 256) { + sgr_set_slot(t, csi->params[0], term_color_index(csi->params[2])); + return; + } + + if (csi->nparams == 6 && csi->params[1] == 5 && csi->params[4] == 5 && + (csi->params[0] == 38 || csi->params[0] == 48) && + csi->params[3] == (csi->params[0] == 38 ? 48 : 38) && + (unsigned)csi->params[2] < 256 && (unsigned)csi->params[5] < 256) { + sgr_set_slot(t, csi->params[0], term_color_index(csi->params[2])); + sgr_set_slot(t, csi->params[3], term_color_index(csi->params[5])); + return; + } + } + + struct term_pen pen = { + .fg = t->cur_fg, .bg = t->cur_bg, .ul = t->cur_ul, + .underline = t->underline, + .bold = t->bold, .faint = t->faint, .italic = t->italic, + .strike = t->strike, .overline = t->overline, .inverse = t->inverse, + }; + sgr_apply(&pen, csi->params, csi->nparams, csi->subparams); + t->cur_fg = pen.fg; + t->cur_bg = pen.bg; + t->cur_ul = pen.ul; + t->underline = pen.underline; + t->bold = pen.bold; + t->faint = pen.faint; + t->italic = pen.italic; + t->strike = pen.strike; + t->overline = pen.overline; + t->inverse = pen.inverse; +} + +#define TERM_PRIV_MODE_FIELDS(X) \ + X(1, application_cursor) \ + X(12, cursor_blink) \ + X(25, cursor_visible) \ + X(67, backarrow_sends_bs) \ + X(1004, focus_events) \ + X(2004, bracketed_paste) \ + X(2026, synchronized_output) + +#define TERM_ANSI_MODE_FIELDS(X) \ + X(4, insert_mode) \ + X(20, lnm) + +static int mode_get(const struct term *t, int mode, bool priv) +{ + if (priv) { + switch (mode) { +#define MODE_FIELD_GET(n, f) case n: return t->f; + TERM_PRIV_MODE_FIELDS(MODE_FIELD_GET) +#undef MODE_FIELD_GET + case 5: return t->reverse_video; + case 6: return t->origin_mode; + case 7: return t->autowrap; + case 47: case 1047: case 1049: return t->alternate; + case 69: return t->lr_margin_mode; + default: break; + } + if (mode_is_mouse(mode)) return t->mouse_mode == mode; + if (mode_is_mouse_enc(mode)) return t->mouse_encoding == mode; + return -1; + } + switch (mode) { +#define MODE_FIELD_GET(n, f) case n: return t->f; + TERM_ANSI_MODE_FIELDS(MODE_FIELD_GET) +#undef MODE_FIELD_GET + } + return -1; +} + +static int mode_status(const struct term *t, int mode, bool priv) +{ + int set = mode_get(t, mode, priv); + return set < 0 ? 0 : set ? 1 : 2; +} + +static int cursor_top(const struct term *t) { return t->origin_mode ? t->scroll_top : 0; } +static int cursor_bottom(const struct term *t) { return t->origin_mode ? t->scroll_bottom : t->rows - 1; } +static int cursor_left(const struct term *t) { return t->origin_mode && t->lr_margin_mode ? t->left_margin : 0; } +static int cursor_right(const struct term *t) { return t->origin_mode && t->lr_margin_mode ? t->right_margin : t->cols - 1; } + +static void mode_set(struct term *t, int mode, bool enabled, bool priv) +{ + if (!priv) { + switch (mode) { +#define MODE_FIELD_SET(n, f) case n: t->f = enabled; break; + TERM_ANSI_MODE_FIELDS(MODE_FIELD_SET) +#undef MODE_FIELD_SET + } + return; + } + switch (mode) { +#define MODE_FIELD_SET(n, f) case n: t->f = enabled; return; + TERM_PRIV_MODE_FIELDS(MODE_FIELD_SET) +#undef MODE_FIELD_SET + default: break; + } + if (mode == 5) { + term_sc_set_reverse_video(t, enabled); + } else if (mode == 6) { + t->origin_mode = enabled; + sc_move(t, 0, enabled ? t->scroll_top : 0); + } else if (mode == 7) { + term_sc_set_autowrap(t, enabled); + } else if (mode == 1048) { + if (enabled) t->saved = term_cursor_save(t); + else sc_restore_cursor(t, &t->saved); + } else if (mode == 47 || mode == 1047 || mode == 1049) { + term_screen_set_alternate(t, enabled); + } else if (mode_is_mouse(mode)) { + t->mouse_mode = enabled ? mode : 0; + } else if (mode == 69) { + t->lr_margin_mode = enabled; + t->left_margin = 0; t->right_margin = t->cols - 1; + sc_move(t, 0, cursor_top(t)); + } else if (mode_is_mouse_enc(mode)) { + t->mouse_encoding = enabled ? mode : 0; + } +} + +/* Vertical moves stop at a margin when they start inside the scroll + * region (xterm, DEC STD 070). */ +static inline void move_v(struct term *t, int n, bool col0) +{ + int top = t->cursor_y >= t->scroll_top ? t->scroll_top : 0; + int bottom = t->cursor_y <= t->scroll_bottom ? t->scroll_bottom : t->rows - 1; + int y = n < 0 ? (t->cursor_y - top >= -n ? t->cursor_y + n : top) + : (n < bottom - t->cursor_y ? t->cursor_y + n : bottom); + sc_move(t, col0 ? cursor_left(t) : t->cursor_x, y); +} +static inline void move_h(struct term *t, int n, bool margin_aware) +{ + int right = margin_aware ? cursor_right(t) : t->cols - 1; + int x = n < 0 ? (t->cursor_x >= -n ? t->cursor_x + n : 0) + : (n < right - t->cursor_x ? t->cursor_x + n : right); + sc_move(t, x, t->cursor_y); +} + +static inline TERM_ALWAYS_INLINE void dispatch_csi(struct term *t, const struct csi *csi) +{ + if (TERM_EXPECT(csi->final == 'm', 1)) { + if (TERM_EXPECT(!csi->prefix && !csi->intermediate, 1)) + handle_sgr(t, csi); + return; + } + if (TERM_EXPECT(csi->final == 'H' || csi->final == 'f', 0)) { + if (TERM_EXPECT(!csi->prefix && !csi->intermediate && !csi->subparams, 1)) { + int64_t r = (int64_t)(csi->params[0] > 0 ? csi->params[0] - 1 : 0) + cursor_top(t); + if (r > cursor_bottom(t)) r = cursor_bottom(t); + int64_t c = (int64_t)(csi->nparams > 1 && csi->params[1] > 0 ? csi->params[1] - 1 : 0) + cursor_left(t); + if (c > cursor_right(t)) c = cursor_right(t); + sc_move(t, (int)c, (int)r); + } + return; + } + + if (csi->intermediate) { + bool plain = (csi->final == 'q' && csi->intermediate == ' ') || + (csi->final == 'p' && csi->intermediate == '!') || + (csi->final == 'x' && csi->intermediate == '*'); + if (plain) { + if (csi->prefix || csi->subparams) return; + } else if (csi->final == 'p' && csi->intermediate == '$') { + if (csi->subparams) return; + } else if (csi->final == 'r' && csi->intermediate == '$') { + if (csi->prefix) return; + } else { + return; + } + } + if (csi->prefix) { + if (csi->final != 'u' && + !(csi->final == 'c' && (csi->prefix == '>' || csi->prefix == '=')) && + !(csi->final == 'q' && csi->prefix == '>' && !csi->intermediate) && + !(csi->final == 'p' && csi->intermediate == '$' && csi->prefix == '?') && + !(csi->final == 'W' && csi->prefix == '?') && + !((csi->final == 'h' || csi->final == 'l') && csi->prefix == '?')) return; + } + if (csi->subparams && !(csi->intermediate == '$' && csi->final == 'r')) return; + switch (csi->final) { + case 'r': { + if (csi->intermediate == '$') { + term_sc_deccara(t, csi); + break; + } + if (csi->prefix || csi->intermediate) break; + int top = csi_count(csi) - 1; + int bottom = csi->nparams > 1 && csi->params[1] ? csi->params[1] - 1 : t->rows - 1; + if (top < bottom && bottom < t->rows) { + t->scroll_top = top; + t->scroll_bottom = bottom; + sc_move(t, 0, t->origin_mode ? top : 0); + } + break; + } + case 'L': + case 'M': + if (csi->prefix || t->cursor_y < t->scroll_top || t->cursor_y > t->scroll_bottom) break; + term_sc_scroll_region(t, t->cursor_y, t->scroll_bottom, csi_count(csi), csi->final == 'L'); + /* IL and DL move the cursor to the left margin (xterm). */ + sc_move(t, t->lr_margin_mode ? t->left_margin : 0, t->cursor_y); + break; + case 'S': + case 'T': + if (csi->prefix || csi->nparams > 1) break; + term_sc_scroll(t, csi_count(csi), csi->final == 'T'); + break; + case 'd': { + int64_t r = (int64_t)csi_count(csi) - 1 + cursor_top(t); + if (r > cursor_bottom(t)) r = cursor_bottom(t); + sc_move(t, t->cursor_x, (int)r); + break; + } + case 'b': + term_text_repeat(t, csi_count(csi)); + break; + case 'A': move_v(t, -csi_count(csi), false); break; + case 'B': move_v(t, csi_count(csi), false); break; + case 'E': move_v(t, csi_count(csi), true); break; + case 'F': move_v(t, -csi_count(csi), true); break; + case 'e': move_v(t, csi_count(csi), false); break; + case 'a': move_h(t, csi_count(csi), true); break; + case 'C': move_h(t, csi_count(csi), false); break; + case 'D': move_h(t, -csi_count(csi), false); break; + case 'G': + case '`': { + int64_t x = (int64_t)csi_count(csi) - 1 + cursor_left(t); + sc_move(t, (int)(x > cursor_right(t) ? cursor_right(t) : x), t->cursor_y); + break; + } + case 's': + if (!csi->prefix && t->lr_margin_mode && csi->nparams > 1) { + int left = csi_count(csi) - 1; + int right = csi->params[1] > 0 ? csi->params[1] - 1 : t->cols - 1; + if (left < right && right < t->cols) { + t->left_margin = left; t->right_margin = right; + sc_move(t, t->origin_mode ? left : 0, cursor_top(t)); + } + } else if (!csi->prefix) { + t->saved = term_cursor_save(t); + } + break; + case 'I': + case 'Z': + term_sc_tab(t, csi->final == 'I' ? csi_count(csi) : -csi_count(csi)); + break; + case 'g': + if (csi->params[0] == 0) t->tabs[t->cursor_x] = 0; + else if (csi->params[0] == 3) term_tabs_clear(t); + break; + case 'W': + if (csi->prefix == '?' && csi->params[0] == 5) { + term_tabs_clear(t); + term_tabs_default(t); + } + break; + case 'n': + if (!csi->prefix && csi->params[0] == 5) send_response(t, "\033[0n"); + else if (!csi->prefix && csi->params[0] == 6) { + /* Report the cursor cell, also with a pending wrap (xterm). */ + char reply[48]; + snprintf(reply, sizeof(reply), "\033[%d;%dR", t->cursor_y - cursor_top(t) + 1, + t->cursor_x - cursor_left(t) + 1); + send_response(t, reply); + } + break; + case 'J': + sc_erase_display(t, csi->params[0]); + break; + case 'X': { + int n = csi_count(csi); + sc_erase_span(t, t->cursor_y, t->cursor_x, + n > t->cols - t->cursor_x ? t->cols : t->cursor_x + n); + break; + } + case 'x': + if (csi->intermediate == '*') + t->decsace = (uint8_t)csi->params[0]; + break; + case 'K': + sc_erase_line(t, csi->params[0]); + break; + case 'P': + sc_delete_chars(t, csi_count(csi)); + break; + case '@': + sc_insert_chars(t, csi_count(csi)); + break; + case 'c': { + if (csi->prefix == '>') + send_response(t, "\x1b[>0;10;0c"); + else if (!csi->prefix && csi->params[0] == 0) + send_response(t, "\x1b[?62;22c"); + break; + } + case 't': + if (!csi->prefix && csi->params[0] == 14) { + char reply[32]; + snprintf(reply, sizeof(reply), "\033[4;%d;%dt", + t->rows * 20, t->cols * 10); + send_response(t, reply); + } + break; + case 'u': { + if (!csi->prefix) { sc_restore_cursor(t, &t->saved); break; } + if (csi->nparams > (csi->prefix == '=' ? 2 : 1)) break; + struct keyboard_mode *k = &t->keyboard[t->alternate]; + unsigned flags = (unsigned)csi->params[0] & 31; + if (csi->prefix == '?') { + char reply[32]; + snprintf(reply, sizeof(reply), "\033[?%uu", k->flags); + send_response(t, reply); + } else if (csi->prefix == '>') { + if (k->depth == KEYBOARD_STACK_SIZE) { + memmove(k->saved, k->saved + 1, sizeof(k->saved) - sizeof(k->saved[0])); + k->depth--; + } + k->saved[k->depth++] = k->flags; + k->flags = flags; + } else if (csi->prefix == '<') { + unsigned count = (unsigned)csi_count(csi); + if (count > k->depth) { k->depth = 0; k->flags = 0; } + else { k->depth -= count; k->flags = k->saved[k->depth]; } + } else if (csi->prefix == '=') { + int mode = csi->nparams > 1 && csi->params[1] ? csi->params[1] : 1; + if (mode == 1) k->flags = flags; + else if (mode == 2) k->flags |= flags; + else if (mode == 3) k->flags &= ~flags; + } + break; + } + case 'h': + case 'l': { + bool enabled = csi->final == 'h'; + bool priv = csi->prefix == '?'; + for (int p = 0; p < csi->nparams; p++) + mode_set(t, csi->params[p], enabled, priv); + break; + } + case 'q': { + if (csi->prefix == '>' && !csi->intermediate && + (!csi->nparams || !csi->params[0])) { + send_response(t, "\033P>|wsap_term\033\\"); + } else if (!csi->prefix && csi->intermediate == ' ') { + term_sc_set_cursor_style(t, csi->nparams > 0 ? csi->params[0] : 0); + } + break; + } + case 'p': + if (csi->intermediate == '!') { + soft_reset(t); + sc_move(t, 0, 0); + } else if (csi->intermediate == '$' && csi->nparams >= 1) { + char reply[32]; + snprintf(reply, sizeof(reply), "\033[%s%d;%d$y", + csi->prefix == '?' ? "?" : "", csi->params[0], + mode_status(t, csi->params[0], csi->prefix == '?')); + send_response(t, reply); + } + break; + default: + break; + } +} + +/* Formats one trace line and hands it to the host's trace callback. */ +static void esc_tracef(const struct term *t, const char *fmt, ...) +{ + char line[512]; + va_list ap; + va_start(ap, fmt); + vsnprintf(line, sizeof line, fmt, ap); + va_end(ap); + t->cb.trace(t->cb.ud, line); +} + +/* The guard keeps formatting off the parser path when tracing is off. */ +#define ESC_TRACE(t, ...) \ + do { if ((t)->cb.trace) esc_tracef((t), __VA_ARGS__); } while (0) + +static void dispatch_esc(struct term *t, unsigned char ch) +{ + t->state = ST_GROUND; + switch (ch) { + case '=': + t->application_keypad = true; + break; + case '>': + t->application_keypad = false; + break; + case 'H': + t->tabs[t->cursor_x] = 1; + break; + case 'c': + if (t->alternate) term_screen_set_alternate(t, false); + soft_reset(t); + term_sc_reset(t); + break; + case '[': + ESC_TRACE(t, "ESC[ → ST_CSI"); + t->state = ST_CSI; + csi_parser_reset(&t->csi_parser); + break; + case ']': + ESC_TRACE(t, "ESC] → ST_OSC"); + t->state = ST_OSC; + t->osc_len = 0; + t->str_truncated = false; + break; + case 'P': + ESC_TRACE(t, "ESCP → ST_DCS"); + t->state = ST_DCS; + t->dcs_len = 0; + t->str_truncated = false; + break; + case '_': + case '^': + case 'X': + t->state = ST_IGNORE; + break; + case '#': + t->state = ST_DECALN; + break; + case '%': + t->state = ST_DOCS; + break; + case '(': + case ')': + case '*': + case '+': + case '-': + case '.': + case '/': + t->charset_designation = (char)ch; + t->state = ST_CHARSET; + break; + case 'D': + case 'E': + if (ch == 'E') sc_cr(t); + sc_linefeed(t, 1); + break; + case 'M': + if (t->cursor_y == t->scroll_top) + term_sc_scroll_region(t, t->scroll_top, t->scroll_bottom, 1, true); + else + sc_move(t, t->cursor_x, t->cursor_y > 0 ? t->cursor_y - 1 : 0); + break; + case '7': + t->saved = term_cursor_save(t); + break; + case '8': + sc_restore_cursor(t, &t->saved); + break; + default: + break; + } +} + +void term_escape_byte(struct term *t, unsigned char ch) +{ + if (ch >= 0x80 && t->state != ST_OSC && t->state != ST_DCS && + t->state != ST_IGNORE) { + ESC_TRACE(t, " 0x%02x in state %d → abort → ST_GROUND", ch, t->state); + t->osc_len = 0; + t->dcs_len = 0; + t->state = ST_GROUND; + term_text_ground(t, ch); + return; + } + switch (t->state) { + case ST_ESC: + ESC_TRACE(t, " ESC+0x%02x('%c')", ch, ch >= 0x20 && ch < 0x7f ? ch : '.'); + if (ch < 0x20) { + t->state = ST_GROUND; + term_text_ground(t, ch); + } else { + dispatch_esc(t, ch); + } + break; + case ST_CSI: + if (ch == 0x1b) { + ESC_TRACE(t, " CSI interrupted by ESC"); + t->state = ST_ESC; + } else if (ch == 0x18 || ch == 0x1a) { + ESC_TRACE(t, " CSI cancelled by 0x%02x", ch); + t->state = ST_GROUND; + } else if (ch < 0x20) { + /* Other C0 controls run inside the sequence (ECMA-48 5.5, + * xterm). */ + term_text_ground(t, ch); + } + break; + case ST_CHARSET: + if (t->charset_designation == '(' || t->charset_designation == ')') { + t->charsets[t->charset_designation == ')'] = (char)ch; + t->graphics = t->charsets[t->charset_active] == '0'; + } + t->state = ST_GROUND; + break; + case ST_DOCS: + if (ch != 'G' && ch != '@' && ch != '/') + term_text_ground(t, '%'); + t->state = ST_GROUND; + if (ch != 'G' && ch != '@' && ch != '/') + term_text_ground(t, ch); + break; + case ST_DECALN: + t->state = ST_GROUND; + if (ch == '8') + term_sc_decaln(t); + break; + case ST_IGNORE: + if (ch == 0x07 || ch == 0x18 || ch == 0x1a) + t->state = ST_GROUND; + else if (ch == 0x1b) + t->state = ST_IGNORE_ESC; + break; + case ST_IGNORE_ESC: + if (ch == '\\' || ch == 0x07 || ch == 0x18 || ch == 0x1a) + t->state = ST_GROUND; + else { + t->state = ST_ESC; + term_escape_byte(t, ch); + } + break; + case ST_OSC: + if (ch == 0x07) { + term_osc_finish(t); + t->state = ST_GROUND; + } else if (ch == 0x1b) { + t->state = ST_STR_ESC; + } else if (ch < 0x20) { + t->osc_len = 0; + t->state = ST_GROUND; + term_text_ground(t, ch); + } else if (term_osc_room(t) || term_osc_grow(t)) { + *term_osc_tail(t) = (char)ch; + term_osc_commit(t, 1); + } else { + t->str_truncated = true; + } + break; + case ST_DCS: + if (ch == 0x07) { + dcs_dispatch(t); + t->dcs_len = 0; + t->state = ST_GROUND; + } else if (ch == 0x1b) { + t->state = ST_DCS_ESC; + } else if (ch < 0x20) { + t->dcs_len = 0; + t->state = ST_GROUND; + term_text_ground(t, ch); + } else if (dcs_reserve(t, 1)) { + t->dcs_buf[t->dcs_len++] = (char)ch; + } else { + t->str_truncated = true; + } + break; + case ST_DCS_ESC: + if (ch == '\\') { + dcs_dispatch(t); + t->dcs_len = 0; + t->state = ST_GROUND; + } else { + t->dcs_len = 0; + t->state = ST_ESC; + term_escape_byte(t, ch); + } + break; + case ST_STR_ESC: + if (ch == '\\') { + term_osc_finish(t); + t->state = ST_GROUND; + } else { + t->osc_len = 0; + t->dcs_len = 0; + t->state = ST_ESC; + term_escape_byte(t, ch); + } + break; + default: + break; + } +} + +static inline size_t str_appendable_span(const char *buf, size_t len) +{ + return scan_printable_span(buf, len, 0x80); +} + +size_t term_escape_str(struct term *t, const char *buf, size_t len) +{ + if (t->state == ST_IGNORE) { + size_t take = str_appendable_span(buf, len); + if (take < len) + term_escape_byte(t, (unsigned char)buf[take++]); + return take; + } + int is_osc = t->state == ST_OSC; + size_t take = str_appendable_span(buf, len); + if (is_osc) { + size_t room = term_osc_room(t); + if (take > room && term_osc_grow(t)) + room = term_osc_room(t); + if (room) { + if (take > room) + take = room; + memcpy(term_osc_tail(t), buf, take); + term_osc_commit(t, take); + } else { + t->str_truncated = true; + } + } else { + dcs_reserve(t, take); + size_t room = t->dcs_cap - (size_t)t->dcs_len; + if (room) { + if (take > room) + take = room; + memcpy(t->dcs_buf + t->dcs_len, buf, take); + t->dcs_len += (int)take; + } else { + t->str_truncated = true; + } + } + if (take < len) + term_escape_byte(t, (unsigned char)buf[take++]); + return take; +} + +size_t term_escape_csi(struct term *t, const char *buf, size_t len) +{ + size_t consumed; + enum csi_result result = csi_read(&t->csi_parser, buf, len, &consumed); + if (result == CSI_CONTROL) { + term_escape_byte(t, (unsigned char)buf[consumed++]); + } else if (result == CSI_DISPATCH || result == CSI_DISCARD) { + if (t->cb.trace) { + const struct csi *c = &t->csi_parser.csi; + char params[256] = "-1"; + if (c->nparams) { + size_t n = 0; + for (int j = 0; j < c->nparams && n < sizeof params; j++) + n += (size_t)snprintf(params + n, sizeof params - n, + j ? ";%d" : "%d", c->params[j]); + } + esc_tracef(t, " CSI complete: %s%c p=[%s] prefix='%c' inter='%c'", + result == CSI_DISCARD ? "INVALID " : "", c->final, params, + c->prefix ? c->prefix : ' ', c->intermediate ? c->intermediate : ' '); + } + if (result == CSI_DISPATCH) + dispatch_csi(t, &t->csi_parser.csi); + t->state = ST_GROUND; + } + return consumed; +} + +void term_escape_begin(struct term *t) +{ + ESC_TRACE(t, "0x1b → ST_ESC"); + t->state = ST_ESC; +} diff --git a/src/escape.h b/src/escape.h new file mode 100644 index 0000000..a66d02c --- /dev/null +++ b/src/escape.h @@ -0,0 +1,10 @@ +#ifndef TERM_ESCAPE_H +#define TERM_ESCAPE_H +#include "internal.h" + +void term_escape_begin(struct term *t); +void term_escape_byte(struct term *t, unsigned char ch); +size_t term_escape_str(struct term *t, const char *buf, size_t len); +size_t term_escape_csi(struct term *t, const char *buf, size_t len); + +#endif diff --git a/src/fastcsi.c b/src/fastcsi.c new file mode 100644 index 0000000..d8983dc --- /dev/null +++ b/src/fastcsi.c @@ -0,0 +1,378 @@ +#include "compat.h" +#include "core.h" +#include "kernels.h" +#include "pen.h" +#include "screen.h" + +#include <stdbool.h> +#include <stddef.h> +#include <stdint.h> +#include <string.h> + +TERM_ALWAYS_INLINE +static inline const char *parse_color_sequence(const char *p, const char *end, uint32_t *color) +{ + const char *s = p + 5; + uint32_t r, g, b; + + if (TERM_EXPECT(s >= end, 0)) return NULL; + unsigned char c0 = (unsigned char)*s++; + unsigned char d0 = (unsigned char)(c0 - '0'); + if (TERM_EXPECT(d0 > 9, 0)) return NULL; + if (TERM_EXPECT(s >= end, 0)) return NULL; + unsigned char c1 = (unsigned char)*s++; + if (c1 == ';') { + r = d0; + } else { + unsigned char d1 = (unsigned char)(c1 - '0'); + if (TERM_EXPECT(d1 > 9, 0)) return NULL; + if (TERM_EXPECT(s >= end, 0)) return NULL; + unsigned char c2 = (unsigned char)*s++; + if (c2 == ';') { + r = d0 * 10 + d1; + } else { + unsigned char d2 = (unsigned char)(c2 - '0'); + if (TERM_EXPECT(d2 > 9 || s >= end || *s++ != ';', 0)) return NULL; + r = d0 * 100 + d1 * 10 + d2; + if (TERM_EXPECT(r > 255, 0)) return NULL; + } + } + + if (TERM_EXPECT(s >= end, 0)) return NULL; + c0 = (unsigned char)*s++; + d0 = (unsigned char)(c0 - '0'); + if (TERM_EXPECT(d0 > 9, 0)) return NULL; + if (TERM_EXPECT(s >= end, 0)) return NULL; + c1 = (unsigned char)*s++; + if (c1 == ';') { + g = d0; + } else { + unsigned char d1 = (unsigned char)(c1 - '0'); + if (TERM_EXPECT(d1 > 9, 0)) return NULL; + if (TERM_EXPECT(s >= end, 0)) return NULL; + unsigned char c2 = (unsigned char)*s++; + if (c2 == ';') { + g = d0 * 10 + d1; + } else { + unsigned char d2 = (unsigned char)(c2 - '0'); + if (TERM_EXPECT(d2 > 9 || s >= end || *s++ != ';', 0)) return NULL; + g = d0 * 100 + d1 * 10 + d2; + if (TERM_EXPECT(g > 255, 0)) return NULL; + } + } + + if (TERM_EXPECT(s >= end, 0)) return NULL; + c0 = (unsigned char)*s++; + d0 = (unsigned char)(c0 - '0'); + if (TERM_EXPECT(d0 > 9, 0)) return NULL; + if (TERM_EXPECT(s >= end, 0)) return NULL; + c1 = (unsigned char)*s++; + if (c1 == 'm') { + b = d0; + } else { + unsigned char d1 = (unsigned char)(c1 - '0'); + if (TERM_EXPECT(d1 > 9, 0)) return NULL; + if (TERM_EXPECT(s >= end, 0)) return NULL; + unsigned char c2 = (unsigned char)*s++; + if (c2 == 'm') { + b = d0 * 10 + d1; + } else { + unsigned char d2 = (unsigned char)(c2 - '0'); + if (TERM_EXPECT(d2 > 9 || s >= end || *s++ != 'm', 0)) return NULL; + b = d0 * 100 + d1 * 10 + d2; + if (TERM_EXPECT(b > 255, 0)) return NULL; + } + } + + *color = term_rgb((int)r, (int)g, (int)b); + return s; +} + +static const char *parse_256color(const char *p, const char *end, uint32_t *color) +{ + const char *q = p + 2; + unsigned v = 0; + int nd = 0; + while (q < end && (unsigned char)(*q - '0') <= 9) { + v = v * 10 + (unsigned char)(*q++ - '0'); + if (++nd > 3) + return NULL; + } + if (!nd || q >= end || *q != 'm' || v > 255) + return NULL; + *color = term_color_index(v); + return q + 1; +} + +TERM_NOINLINE +int term_try_fast_csi(struct term *t, const char *buf, size_t rem) +{ + const char *end = buf + rem; + const char *p = buf; + const char *esc = buf; + const int cols = t->cols; + const int rows = t->rows; + uint32_t fg = t->cur_fg; + uint32_t bg = t->cur_bg; + uint32_t flags_pad = 0; + bool flags_ok = false; + + while (p < end) { + if (p + 5 <= end && (p[0] == '3' || p[0] == '4') && p[1] == '8' && + p[2] == ';' && (p[3] == '2' || p[3] == '5') && p[4] == ';') { + bool is_fg = p[0] == '3'; + uint32_t *slot = is_fg ? &fg : &bg; + uint32_t *other = is_fg ? &bg : &fg; + const char *next = p[3] == '2' ? parse_color_sequence(p, end, slot) + : parse_256color(p + 3, end, slot); + if (!next) { + p = esc; + break; + } + p = next; + if (p + 7 <= end && p[0] == '\033' && p[1] == '[' && + p[2] == (is_fg ? '4' : '3') && p[3] == '8' && + p[4] == ';' && (p[5] == '2' || p[5] == '5') && p[6] == ';') { + next = p[5] == '2' ? parse_color_sequence(p + 2, end, other) + : parse_256color(p + 5, end, other); + if (!next) break; + p = next; + } + if (p < end && (unsigned char)*p >= 0x20 && (unsigned char)*p < 0x7f) { + char ch = *p++; + if (TERM_EXPECT(!flags_ok, 0)) { + flags_pad = term_flags_pad(t); + flags_ok = true; + } + if (TERM_EXPECT(!sc_put1_fast(t, ch, fg, bg, flags_pad), 0)) { + t->cur_fg = fg; + t->cur_bg = bg; + sc_print_ascii(t, &ch, 1); + } + if (p + 2 <= end && p[0] == '\033' && p[1] == '[') { + esc = p; + p += 2; + continue; + } + } + } else if (p[0] == 'm' || (p[0] == '0' && p + 1 < end && p[1] == 'm')) { + reset_sgr(t); + fg = t->cur_fg; + bg = t->cur_bg; + /* Recompute instead of zeroing. The cached flags carry the link + * id, and reset_sgr leaves cur_link set, so zeroing here would + * drop the hyperlink from every cell written after the reset. */ + flags_pad = term_flags_pad(t); + flags_ok = true; + p += p[0] == 'm' ? 1 : 2; + } else if (p[0] == 'J' || + (p[0] >= '0' && p[0] <= '3' && p + 1 < end && p[1] == 'J')) { + t->cur_fg = fg; + t->cur_bg = bg; + sc_erase_display(t, p[0] == 'J' ? 0 : p[0] - '0'); + p += p[0] == 'J' ? 1 : 2; + } else if (p[0] == 'K' || + (p[0] >= '0' && p[0] <= '2' && p + 1 < end && p[1] == 'K')) { + t->cur_fg = fg; + t->cur_bg = bg; + sc_erase_line(t, p[0] == 'K' ? 0 : p[0] - '0'); + p += p[0] == 'K' ? 1 : 2; + } else if (!t->origin_mode && !t->lr_margin_mode && + ((*p >= '0' && *p <= '9') || *p == ';' || + *p == 'A' || *p == 'B' || *p == 'C' || *p == 'D' || + *p == 'E' || *p == 'F' || *p == 'G' || *p == '`' || + *p == 'a' || *p == 'd' || *p == 'e' || + *p == 'H' || *p == 'f' || *p == 'm' || + *p == 'J' || *p == 'K' || *p == 'X' || + *p == 'P' || *p == '@' || *p == 's' || *p == 'u')) { + const char *q = p; + int pv[12] = { 0 }; + int np = 0; + while (q < end && ((*q >= '0' && *q <= '9') || *q == ';')) { + if (*q == ';') { + if (np == 11) { q = NULL; break; } + np++; + q++; + continue; + } + if (pv[np] < 100000) + pv[np] = pv[np] * 10 + (*q - '0'); + q++; + } + if (!q || q >= end) { + p = esc; + break; + } + int nparams = q > p ? np + 1 : 0; + int n = pv[0] > 0 ? pv[0] : 1; + int nx = t->cursor_x, ny = t->cursor_y; + switch (*q) { + case 'A': ny -= n; goto vmove; + case 'B': case 'e': ny += n; goto vmove; + case 'E': ny += n; nx = 0; goto vmove; + case 'F': ny -= n; nx = 0; goto vmove; + vmove: + if (TERM_EXPECT(t->scroll_top != 0 || t->scroll_bottom != rows - 1, 0)) { + p = esc; + goto done; + } + goto move; + case 'C': case 'a': nx += n; goto move; + case 'D': nx -= n; goto move; + case 'G': case '`': nx = n - 1; goto move; + case 'd': ny = n - 1; goto move; + case 'H': case 'f': + ny = n - 1; + nx = np && pv[1] > 0 ? pv[1] - 1 : 0; + goto move; + move: + sc_move(t, nx, ny); + p = q + 1; + break; + case 'm': { + for (int j = 0; j < nparams; j++) { + int v = pv[j]; + switch (v) { + case 0: reset_sgr(t); fg = t->cur_fg; bg = t->cur_bg; break; + case 1: t->bold = true; break; + case 2: t->faint = true; break; + case 3: t->italic = true; break; + case 4: t->underline = 1; break; + case 7: t->inverse = true; break; + case 9: t->strike = true; break; + case 21: t->underline = 2; break; + case 22: t->bold = false; t->faint = false; break; + case 23: t->italic = false; break; + case 24: t->underline = 0; break; + case 27: t->inverse = false; break; + case 29: t->strike = false; break; + case 38: case 48: case 58: { + uint32_t *slot = v == 38 ? &fg : v == 48 ? &bg : &t->cur_ul; + if (j + 2 < nparams && pv[j + 1] == 5) { + if (pv[j + 2] >= 0 && pv[j + 2] <= 255) + *slot = term_color_index(pv[j + 2]); + j += 2; + } else if (j + 1 < nparams && pv[j + 1] == 2 && + j + 4 < nparams) { + *slot = term_rgb(pv[j + 2], pv[j + 3], pv[j + 4]); + j += 4; + } + break; + } + case 39: fg = TERM_COLOR_FG; break; + case 49: bg = TERM_COLOR_BG; break; + case 53: t->overline = true; break; + case 55: t->overline = false; break; + case 59: t->cur_ul = 0; break; + default: + if ((unsigned)(v - 30) < 8) fg = term_color_index(v - 30); + else if ((unsigned)(v - 40) < 8) bg = term_color_index(v - 40); + else if ((unsigned)(v - 90) < 8) fg = term_color_index(v - 90 + 8); + else if ((unsigned)(v - 100) < 8) bg = term_color_index(v - 100 + 8); + break; + } + } + flags_ok = false; + p = q + 1; + break; + } + case 'J': + t->cur_fg = fg; + t->cur_bg = bg; + sc_erase_display(t, pv[0]); + p = q + 1; + break; + case 'K': + t->cur_fg = fg; + t->cur_bg = bg; + sc_erase_line(t, pv[0]); + p = q + 1; + break; + case 'X': + t->cur_fg = fg; + t->cur_bg = bg; + sc_erase_span(t, t->cursor_y, t->cursor_x, + n > cols - t->cursor_x ? cols : t->cursor_x + n); + p = q + 1; + break; + case 'P': + t->cur_fg = fg; + t->cur_bg = bg; + sc_delete_chars(t, n); + p = q + 1; + break; + case '@': + t->cur_fg = fg; + t->cur_bg = bg; + sc_insert_chars(t, n); + p = q + 1; + break; + case 's': + t->cur_fg = fg; + t->cur_bg = bg; + t->saved = term_cursor_save(t); + p = q + 1; + break; + case 'u': + t->cur_fg = fg; + t->cur_bg = bg; + sc_restore_cursor(t, &t->saved); + fg = t->cur_fg; + bg = t->cur_bg; + flags_ok = false; + p = q + 1; + if (t->origin_mode || t->graphics) + goto done; + break; + default: + p = esc; + goto done; + } + } else { + p = esc; + break; + } +tail: + if (p < end && (unsigned char)(*p - 0x20) < 0x5f && !t->pending_wrap) { + t->cur_fg = fg; + t->cur_bg = bg; + int avail = cols - t->cursor_x; + int limit = end - p < (ptrdiff_t)avail ? (int)(end - p) : avail; + int run = scan_ascii_run(p, limit); + if (TERM_EXPECT(!flags_ok, 0)) { + flags_pad = term_flags_pad(t); + flags_ok = true; + } + if (!sc_ascii_fast(t, p, run, fg, bg, flags_pad)) + sc_print_ascii(t, p, run); + p += run; + } + if (p + 2 <= end && p[0] == '\033' && p[1] == '[') { + esc = p; + p += 2; + continue; + } + if (p < end && *p == '\r') { + sc_cr(t); + p++; + goto tail; + } + if (p < end && *p == '\n') { + t->cur_fg = fg; + t->cur_bg = bg; + if (term_lnm(t)) sc_cr(t); + sc_linefeed(t, 1); + p++; + goto tail; + } + break; + } +done: + if (p == end && esc != buf && esc == p - 2) + p = esc; + t->cur_fg = fg; + t->cur_bg = bg; + if (p > buf) + return (int)(p - buf); + return 0; +} diff --git a/src/gstore.c b/src/gstore.c new file mode 100644 index 0000000..c5109c6 --- /dev/null +++ b/src/gstore.c @@ -0,0 +1,380 @@ +#include "gstore.h" +#include "internal.h" +#include "screen.h" +#include "unicode/grapheme.h" + +#include <stdlib.h> +#include <string.h> + +#define GSTORE_MAX_ENTRIES 0x00FFFFFFu + +struct gentry { + union { + uint32_t offset; /* Offset into arena for code points past index 1 */ + uint32_t next_free; /* Next entry in free list (1-based, 0 = none) */ + }; + uint16_t len; /* Number of code points */ + bool mark; /* Reachable in current GC sweep */ + bool allocated; /* True if actively holding a cluster */ + uint32_t cp0; /* First code point (inlined) */ + uint32_t cp1; /* Second code point (inlined) */ +}; + +_Static_assert(sizeof(struct gentry) == 16, + "struct gentry must stay 16 bytes (4 entries per cache line)"); + +struct gslot { + uint32_t hash; /* 32-bit hash of cluster code points */ + uint32_t entry_idx; /* 1-based index into g->entries (0 = empty) */ +}; + +struct gstore { + struct term *term; + uint32_t *arena; + size_t arena_len; + size_t arena_cap; + + struct gentry *entries; + uint32_t entries_count; + uint32_t entries_cap; + uint32_t free_head; /* 1-based index, 0 = none */ + + uint32_t live_count; + uint32_t last_live; + uint32_t sweep_threshold; + uint32_t pin_depth; + + struct gslot *slots; + size_t slot_count; +}; + +static uint64_t hash_cps(const uint32_t *cps, int len) +{ + uint64_t h = HASH64_BASIS; + for (int i = 0; i < len; i++) + h = hash64_step(h, cps[i]); + return hash64_finish(h); +} + +static inline bool gentry_matches(const struct gstore *g, const struct gentry *e, + const uint32_t *cps, int n) +{ + if ((int)e->len != n) + return false; + if (e->cp0 != cps[0] || e->cp1 != cps[1]) + return false; + if (n == 2) + return true; + return !memcmp(&g->arena[e->offset], &cps[2], (size_t)(n - 2) * sizeof(*cps)); +} + +int term_gstore_expand(const struct gstore *g, uint32_t handle, uint32_t *cps, int cap) +{ + if (cap <= 0) + return 0; + if (!g || !grapheme_is_handle(handle)) { + cps[0] = handle; + return 1; + } + uint32_t idx = handle - GRAPHEME_HANDLE_BASE; + if (idx >= g->entries_count || !g->entries[idx].allocated) { + cps[0] = 0xFFFD; + return 1; + } + const struct gentry *e = &g->entries[idx]; + int n = (int)e->len; + if (n > cap) { + cps[0] = 0xFFFD; + return 1; + } + cps[0] = e->cp0; + cps[1] = e->cp1; + if (n > 2) + memcpy(&cps[2], &g->arena[e->offset], (size_t)(n - 2) * sizeof(*cps)); + return n; +} + +static void gstore_insert_slot(struct gstore *g, uint32_t h32, uint32_t entry_idx) +{ + uint32_t mask = (uint32_t)g->slot_count - 1; + uint32_t idx = h32 & mask; + for (uint32_t probe = 0; probe <= mask; probe++) { + struct gslot *slot = &g->slots[idx]; + if (slot->entry_idx == 0) { + slot->hash = h32; + slot->entry_idx = entry_idx + 1; + return; + } + idx = (idx + 1) & mask; + } +} + +static bool grow_slots(struct gstore *g) +{ + size_t want = g->slot_count ? g->slot_count * 2 : 256; + struct gslot *next = calloc(want, sizeof(*next)); + if (!next) + return false; + + free(g->slots); + g->slots = next; + g->slot_count = want; + + for (uint32_t i = 0; i < g->entries_count; i++) { + struct gentry *e = &g->entries[i]; + if (!e->allocated) + continue; + uint32_t cps_buf[GRAPHEME_MAX_LEN]; + int n = term_gstore_expand(g, GRAPHEME_HANDLE_BASE + i, cps_buf, GRAPHEME_MAX_LEN); + uint64_t h = hash_cps(cps_buf, n); + uint32_t h32 = (uint32_t)h ? (uint32_t)h : 1; + gstore_insert_slot(g, h32, i); + } + return true; +} + +static uint32_t gstore_find(const struct gstore *g, const uint32_t *cps, int n, uint32_t h32) +{ + if (!g->slots || g->slot_count == 0) + return 0; + uint32_t mask = (uint32_t)g->slot_count - 1; + uint32_t idx = h32 & mask; + for (uint32_t probe = 0; probe <= mask; probe++) { + const struct gslot *slot = &g->slots[idx]; + if (slot->entry_idx == 0) + return 0; + if (slot->hash == h32) { + uint32_t e_idx = slot->entry_idx - 1; + const struct gentry *e = &g->entries[e_idx]; + if (e->allocated && gentry_matches(g, e, cps, n)) + return GRAPHEME_HANDLE_BASE + e_idx; + } + idx = (idx + 1) & mask; + } + return 0; +} + +struct gstore *term_gstore_new(struct term *t) +{ + struct gstore *g = calloc(1, sizeof(*g)); + if (!g) + return NULL; + g->term = t; + g->sweep_threshold = 1024; + return g; +} + +void term_gstore_free(struct gstore *g) +{ + if (!g) + return; + free(g->arena); + free(g->entries); + free(g->slots); + free(g); +} + +uint32_t term_gstore_count(const struct gstore *g) +{ + return g ? g->live_count : 0; +} + +void term_gstore_pin(struct gstore *g) +{ + if (g) + g->pin_depth++; +} + +void term_gstore_unpin(struct gstore *g) +{ + if (g && g->pin_depth > 0) + g->pin_depth--; +} + +void term_gstore_mark(struct gstore *g, uint32_t handle) +{ + if (!g || !grapheme_is_handle(handle)) + return; + uint32_t idx = handle - GRAPHEME_HANDLE_BASE; + if (idx < g->entries_count && g->entries[idx].allocated) + g->entries[idx].mark = true; +} + +static void sweep_mark_row(struct gstore *g, const struct cell *cells, int len) +{ + for (int x = 0; x < len; x++) { + uint32_t c = cells[x].c; + if (grapheme_is_handle(c)) + term_gstore_mark(g, c); + } +} + +void term_gstore_sweep(struct gstore *g, struct term *t) +{ + if (!g) + return; + + if (t) { + if (grapheme_is_handle(t->last_char)) + term_gstore_mark(g, t->last_char); + + struct screen *s = &t->scr; + for (int y = 0; y < s->rows; y++) { + struct srow *row = screen_get_row(s, y); + if (row && row->kind == ROW_CELLS && row->len > 0) + sweep_mark_row(g, row->cells, (int)row->len); + } + if (t->alternate && t->alt_allocated) { + struct screen *as = &t->alt_scr; + for (int y = 0; y < as->rows; y++) { + struct srow *row = screen_get_row(as, y); + if (row && row->kind == ROW_CELLS && row->len > 0) + sweep_mark_row(g, row->cells, (int)row->len); + } + } + } + + /* Compact the code point arena for clusters with n > 2. */ + uint32_t new_arena_len = 0; + uint32_t *packed = g->arena ? malloc(g->arena_cap * sizeof(*packed)) : NULL; + if (packed) { + for (uint32_t i = 0; i < g->entries_count; i++) { + struct gentry *e = &g->entries[i]; + if (e->allocated && e->mark && e->len > 2) { + size_t extra = (size_t)(e->len - 2); + memcpy(&packed[new_arena_len], &g->arena[e->offset], + extra * sizeof(*packed)); + e->offset = new_arena_len; + new_arena_len += (uint32_t)extra; + } + } + free(g->arena); + g->arena = packed; + g->arena_len = new_arena_len; + } + + /* Rebuild open-addressing slot table and free list. */ + g->free_head = 0; + uint32_t live = 0; + if (g->slots) + memset(g->slots, 0, g->slot_count * sizeof(*g->slots)); + + for (uint32_t i = 0; i < g->entries_count; i++) { + struct gentry *e = &g->entries[i]; + if (e->allocated) { + if (e->mark) { + live++; + e->mark = false; + if (g->slots) { + uint32_t cps_buf[GRAPHEME_MAX_LEN]; + int n = term_gstore_expand(g, GRAPHEME_HANDLE_BASE + i, cps_buf, GRAPHEME_MAX_LEN); + uint64_t h = hash_cps(cps_buf, n); + uint32_t h32 = (uint32_t)h ? (uint32_t)h : 1; + gstore_insert_slot(g, h32, i); + } + } else { + e->allocated = false; + e->len = 0; + e->offset = 0; + e->cp0 = 0; + e->cp1 = 0; + e->next_free = g->free_head; + g->free_head = i + 1; + } + } else { + e->next_free = g->free_head; + g->free_head = i + 1; + } + } + + g->live_count = live; + g->last_live = live; + g->sweep_threshold = live * 2; + if (g->sweep_threshold < 1024) + g->sweep_threshold = 1024; +} + +uint32_t term_gstore_intern(struct gstore *g, const uint32_t *cps, int n) +{ + if (!g || n <= 0) + return 0; + if (n == 1) + return cps[0]; + if (n > GRAPHEME_MAX_LEN) + n = GRAPHEME_MAX_LEN; + + uint64_t h = hash_cps(cps, n); + uint32_t h32 = (uint32_t)h ? (uint32_t)h : 1; + + uint32_t existing = gstore_find(g, cps, n, h32); + if (existing) + return existing; + + if (g->term && g->pin_depth == 0 && + (g->live_count >= g->sweep_threshold || + g->live_count >= GSTORE_MAX_ENTRIES)) { + term_gstore_sweep(g, g->term); + existing = gstore_find(g, cps, n, h32); + if (existing) + return existing; + } + + if (!g->slots || (g->live_count + 1) * 2 > g->slot_count) { + if (!grow_slots(g) && !g->slots) + return cps[0]; + } + + uint32_t entry_idx; + if (g->free_head) { + entry_idx = g->free_head - 1; + g->free_head = g->entries[entry_idx].next_free; + } else { + if (g->entries_count >= GSTORE_MAX_ENTRIES) + return cps[0]; + if (g->entries_count == g->entries_cap) { + size_t new_cap = g->entries_cap ? (size_t)g->entries_cap * 2 : 128; + if (new_cap > GSTORE_MAX_ENTRIES) + new_cap = GSTORE_MAX_ENTRIES; + struct gentry *next = realloc(g->entries, new_cap * sizeof(*next)); + if (!next) + return cps[0]; + g->entries = next; + g->entries_cap = (uint32_t)new_cap; + } + entry_idx = g->entries_count++; + } + + struct gentry *e = &g->entries[entry_idx]; + e->cp0 = cps[0]; + e->cp1 = cps[1]; + e->len = (uint16_t)n; + e->mark = false; + e->allocated = true; + e->offset = 0; + + if (n > 2) { + size_t extra = (size_t)(n - 2); + if (g->arena_len + extra > g->arena_cap) { + size_t new_cap = g->arena_cap ? g->arena_cap * 2 : 512; + while (new_cap < g->arena_len + extra) + new_cap *= 2; + uint32_t *next = realloc(g->arena, new_cap * sizeof(*next)); + if (!next) { + e->allocated = false; + e->next_free = g->free_head; + g->free_head = entry_idx + 1; + return cps[0]; + } + g->arena = next; + g->arena_cap = new_cap; + } + uint32_t off = (uint32_t)g->arena_len; + memcpy(&g->arena[off], &cps[2], extra * sizeof(*cps)); + g->arena_len += extra; + e->offset = off; + } + + gstore_insert_slot(g, h32, entry_idx); + g->live_count++; + return GRAPHEME_HANDLE_BASE + entry_idx; +} diff --git a/src/gstore.h b/src/gstore.h new file mode 100644 index 0000000..0cedd30 --- /dev/null +++ b/src/gstore.h @@ -0,0 +1,36 @@ +#ifndef WSAP_TERM_GSTORE_H +#define WSAP_TERM_GSTORE_H + +#include <stdbool.h> +#include <stddef.h> +#include <stdint.h> + +struct gstore; +struct term; + +/* Create a grapheme store for a terminal. */ +struct gstore *term_gstore_new(struct term *t); + +/* Free the grapheme store and its memory. */ +void term_gstore_free(struct gstore *g); + +/* Store a cluster of code points. Return its handle. */ +uint32_t term_gstore_intern(struct gstore *g, const uint32_t *cps, int n); + +/* Expand a handle to its code points. Return the count. */ +int term_gstore_expand(const struct gstore *g, uint32_t handle, uint32_t *cps, int cap); + +/* Mark reachable clusters and free unused ones. */ +void term_gstore_sweep(struct gstore *g, struct term *t); + +/* Mark one cluster handle as reachable. */ +void term_gstore_mark(struct gstore *g, uint32_t handle); + +/* Return the count of active clusters in the store. */ +uint32_t term_gstore_count(const struct gstore *g); + +/* Suppress automatic GC sweeps while temporary cell buffers hold handles. */ +void term_gstore_pin(struct gstore *g); +void term_gstore_unpin(struct gstore *g); + +#endif diff --git a/src/hist.c b/src/hist.c new file mode 100644 index 0000000..123ed7b --- /dev/null +++ b/src/hist.c @@ -0,0 +1,3256 @@ +#include "hist.h" +#include "compat.h" +#include "kernels.h" +#include "unicode/grapheme.h" + +#include <assert.h> +#include <stdlib.h> +#include <string.h> +#include <stdio.h> + +/* + * Text format. A NARROW part is raw ASCII, one byte per cell. Other parts are + * UTF-8 with prefix bytes that UTF-8 never uses: + * 0xF5 cp wide + * 0xF6 cp wide, emoji presentation + * 0xF7 cp narrow, text presentation + * 0xF8 k cp... cluster of k & 0x1f code points; k & 0x80 wide, + * k & 0x40 emoji, k & 0x20 text presentation + * Trail cells and wrap pads are not stored. + * + * Attribute runs follow the text. A run is a header byte that names the + * fields that differ from the previous run, a LEB128 cell count, then those + * fields. Cells after the last run use the default pen. + */ + +#define RUN_FG (1u << 0) +#define RUN_BG (1u << 1) +#define RUN_ATTR (1u << 2) +#define RUN_LINK (1u << 3) +#define RUN_MAX 18 + +#define PREFIX_WIDE 0xF5 +#define PREFIX_EMOJI 0xF6 +#define PREFIX_TEXT 0xF7 +#define PREFIX_CLUSTER 0xF8 + +#define STYLE_FLAGS (CELL_BOLD | CELL_ITALIC | CELL_ANY_UNDERLINE | CELL_STRIKE | \ + CELL_OVERLINE | CELL_FAINT | CELL_LINK) +#define SHAPE_FLAGS (CELL_WIDE | CELL_TRAIL | CELL_WRAP_PAD | CELL_EMOJI | CELL_TEXT) + +#define NO_TEXT UINT32_MAX +#define ROWS_UNKNOWN 255 +/* Bytes one cell can need, except a cluster. */ +#define CELL_BYTES 5 +#define CLUSTER_BYTES (2 + GRAPHEME_MAX_LEN * 4) + +/* ---- Parts and pages ---- */ + +static inline struct hline *part(const struct hist *h, uint64_t seq) +{ + return &h->ring[seq & h->ring_mask]; +} + +static inline uint8_t *part_text(const struct hist *h, const struct hline *p) +{ + return h->pages[p->off >> 16] + (p->off & 0xffff); +} + +static inline struct hline *open_part(const struct hist *h) +{ + return hist_is_open(h) ? part(h, h->head - 1) : NULL; +} + +/* Grow the ring so n more parts fit. The caller checks the fast path first. */ +static TERM_NOINLINE bool ring_grow(struct hist *h, uint64_t n) +{ + uint64_t cap = h->ring ? (uint64_t)h->ring_mask + 1 : 65536; + if (cap < 65536) + cap = 65536; + while (cap < h->head - h->tail + n) + cap *= 2; + struct hline *ring = term_aligned_alloc(64, (size_t)cap * sizeof(*ring)); + if (!ring) + return false; + struct hist_block *blocks = calloc((size_t)cap / 16, sizeof(*blocks)); + if (!blocks) { + term_aligned_free(ring); + return false; + } + uint32_t *exact = h->exact_rows ? calloc((size_t)cap, sizeof(*exact)) : NULL; + if (h->exact_rows && !exact) { + free(blocks); + term_aligned_free(ring); + return false; + } + for (uint64_t s = h->tail; s < h->head; s++) { + ring[s & (cap - 1)] = *part(h, s); + if (exact) + exact[s & (cap - 1)] = h->exact_rows[s & h->ring_mask]; + } + free(h->exact_rows); + h->exact_rows = exact; + term_aligned_free(h->ring); + free(h->blocks); + h->blocks = blocks; + h->ring = ring; + h->ring_mask = (uint32_t)(cap - 1); + return true; +} + +/* Room for n more parts. The check runs on the write path per batch, so + * keep it inline and leave only the rehash to a call. */ +static inline bool ring_fit(struct hist *h, uint64_t n) +{ + if (TERM_EXPECT(h->ring && h->head - h->tail + n <= (uint64_t)h->ring_mask + 1, 1)) + return true; + return ring_grow(h, n); +} + +static inline bool ring_reserve(struct hist *h) +{ + return ring_fit(h, 1); +} + +static inline uint64_t cont_at(const struct hist *h, uint32_t i) +{ + return h->cont[(h->cont_first + i) & h->cont_mask]; +} + +static bool cont_push(struct hist *h, uint64_t seq) +{ + if (!h->cont || h->cont_count > h->cont_mask) { + uint32_t cap = h->cont ? (h->cont_mask + 1) * 2 : 16; + uint64_t *cont = malloc((size_t)cap * sizeof(*cont)); + if (!cont) + return false; + for (uint32_t i = 0; i < h->cont_count; i++) + cont[i] = cont_at(h, i); + free(h->cont); + h->cont = cont; + h->cont_mask = cap - 1; + h->cont_first = 0; + } + h->cont[(h->cont_first + h->cont_count++) & h->cont_mask] = seq; + return true; +} + +/* Keep the CONT list to the parts in (tail, head). */ +static void cont_trim(struct hist *h) +{ + while (h->cont_count && cont_at(h, 0) <= h->tail) { + h->cont_first = (h->cont_first + 1) & h->cont_mask; + h->cont_count--; + } + while (h->cont_count && cont_at(h, h->cont_count - 1) >= h->head) + h->cont_count--; +} + +/* The first part of line id. Line starts are the parts without CONT, and the + * tail part, which always starts the base line. */ +static uint64_t line_first(const struct hist *h, uint64_t id) +{ + uint64_t want = h->tail + (id - h->base_id); + if (id == h->base_id || !h->cont_count) + return want; + uint32_t lo = 0, hi = h->cont_count; + while (lo < hi) { + uint32_t mid = lo + (hi - lo) / 2; + if (cont_at(h, mid) - mid <= want) + lo = mid + 1; + else + hi = mid; + } + return want + lo; +} + +static inline uint64_t line_end(const struct hist *h, uint64_t first) +{ + uint64_t s = first + 1; + while (s < h->head && (part(h, s)->flags & HLINE_CONT)) + s++; + return s; +} + +static inline bool line_valid(const struct hist *h, uint64_t id) +{ + return id >= h->base_id && id < h->base_id + h->lines; +} + +static TERM_ALWAYS_INLINE inline void page_drop_ref(struct hist *h, uint32_t pg) +{ + if (--h->page_parts[pg] == 0) { + if (pg == h->cur_page) + h->page_used[pg] = 0; + else + h->free_pages[h->nfree++] = (uint16_t)pg; + } +} + +/* Take a counted line out of counted_rows. Call it while the line is whole. */ +static void rows_uncount(struct hist *h, uint64_t id) +{ + if (id < h->rows_id) + h->counted_rows -= term_hist_line_estimate(h, id, h->wrap_cols); +} + +static void line_gone(struct hist *h) +{ + h->base_id++; + h->lines--; + if (h->rows_id < h->base_id) + h->rows_id = h->base_id; +} + +/* Drop the tail part. A line whose later parts remain keeps its id. */ +static bool evict_part(struct hist *h) +{ + if (h->tail == h->head) + return false; + struct hline *p = part(h, h->tail); + if (p->flags & HLINE_OPEN) + return false; + bool cut = h->tail + 1 < h->head && (part(h, h->tail + 1)->flags & HLINE_CONT); + if (cut) { + /* The line loses its head: count every line again. */ + h->index_epoch++; + if (h->base_cell_id != h->base_id) { + h->base_cell_id = h->base_id; + h->base_cell = 0; + } + h->base_cell += p->cells; + if (h->lines == 1) { + h->last_cells -= p->cells; + h->last_wide -= p->wide; + } + h->rows_id = h->base_id; + h->counted_rows = 0; + h->walk_on = false; + } else { + rows_uncount(h, h->base_id); + } + if (p->off != NO_TEXT) + page_drop_ref(h, p->off >> 16); + h->tail++; + if (cut) + cont_trim(h); + else + line_gone(h); + return true; +} + +static TERM_NOINLINE void evict_line_parts(struct hist *h) +{ + uint64_t end = line_end(h, h->tail); + if (part(h, end - 1)->flags & HLINE_OPEN) + return; + rows_uncount(h, h->base_id); + for (uint64_t s = h->tail; s < end; s++) + if (part(h, s)->off != NO_TEXT) + page_drop_ref(h, part(h, s)->off >> 16); + h->tail = end; + cont_trim(h); + line_gone(h); +} + +static void evict_line(struct hist *h); + +/* Drop n refs to one page at once. Same effect as n calls to + * page_drop_ref, because the count cannot pass zero. */ +static inline void page_drop_n(struct hist *h, uint32_t pg, uint32_t n) +{ + if ((h->page_parts[pg] -= n) == 0) { + if (pg == h->cur_page) + h->page_used[pg] = 0; + else + h->free_pages[h->nfree++] = (uint16_t)pg; + } +} + +#if defined(TERM_KERNEL_AVX512) +TERM_TARGET_AVX512 +static inline bool same_page_narrow_rows_avx512(const struct hline *p, uint32_t page) +{ + const __m512i pg = _mm512_set1_epi32((int)(page << 16)); + const __m512i mask = _mm512_set1_epi32((int)0xffff0000u); + const __m512i tag = _mm512_set1_epi32(0x01010000); + for (int i = 0; i < 16; i += 4) { + __m512i v = _mm512_loadu_si512((const void *)(p + i)); + if ((_mm512_cmpeq_epi32_mask(v, tag) & 0x8888) != 0x8888 || + (_mm512_cmpeq_epi32_mask(_mm512_and_si512(v, mask), pg) & 0x1111) != 0x1111) + return false; + } + return true; +} +#endif + +#if defined(TERM_KERNEL_AVX2) +TERM_TARGET_AVX2 +static inline bool same_page_narrow_rows_avx2(const struct hline *p, uint32_t page) +{ + const __m256i pg = _mm256_set1_epi32((int)(page << 16)); + const __m256i mask = _mm256_set1_epi32((int)0xffff0000u); + const __m256i tag = _mm256_set1_epi32(0x01010000); + for (int i = 0; i < 16; i += 2) { + __m256i v = _mm256_loadu_si256((const __m256i *)(p + i)); + if ((_mm256_movemask_ps(_mm256_castsi256_ps(_mm256_cmpeq_epi32(v, tag))) & 0x88) != 0x88 || + (_mm256_movemask_ps(_mm256_castsi256_ps(_mm256_cmpeq_epi32(_mm256_and_si256(v, mask), pg))) & 0x11) != 0x11) + return false; + } + return true; +} +#endif + +/* Check 16 narrow rows. Each descriptor has four words. */ +static inline bool same_page_narrow_rows(const struct hline *p, uint32_t page) +{ +#if defined(TERM_KERNEL_AVX512) + if (term_use_avx512()) + return same_page_narrow_rows_avx512(p, page); +#endif +#if defined(TERM_KERNEL_AVX2) + if (term_use_avx2()) + return same_page_narrow_rows_avx2(p, page); +#endif +#if defined(__aarch64__) + const uint32x4_t expected = {page << 16, 0, 0, 0x01010000}; + const uint32x4_t mask = {0xffff0000u, 0, 0, 0xffffffffu}; + for (int i = 0; i < 16; i++) { + uint32x4_t v = vreinterpretq_u32_u8(vld1q_u8((const uint8_t *)(p + i))); + if (vminvq_u32(vceqq_u32(vandq_u32(v, mask), expected)) != UINT32_MAX) + return false; + } + return true; +#else + for (int i = 0; i < 16; i++) { + if (p[i].off == NO_TEXT || (p[i].off >> 16) != page || + p[i].flags != HLINE_NARROW || p[i].attr_len || p[i].wrap_rows != 1) + return false; + } + return true; +#endif +} + +#if defined(TERM_KERNEL_AVX512) +TERM_TARGET_AVX512 +static inline bool same_page_rows_64(const struct hline *p, uint32_t page, + uint32_t *text_count, uint32_t *row_count) +{ + const __m512i pg = _mm512_set1_epi32((int)(page << 16)); + const __m512i mask = _mm512_set1_epi32((int)0xffff0000u); + const __m512i open = _mm512_set1_epi32(0x00300000); + const __m512i unknown = _mm512_set1_epi32((int)0xff000000u); + const __m512i no_text = _mm512_set1_epi32(-1); + __m512i sums = _mm512_setzero_si512(); + __mmask16 tag_fail = 0, pg_fail = 0; + uint32_t blanks = 0; + for (int i = 0; i < 64; i += 4) { + __m512i v = _mm512_loadu_si512((const void *)(p + i)); + __mmask16 empty = _mm512_cmpeq_epi32_mask(v, no_text) & 0x1111; + blanks += term_popcount32(empty); + tag_fail |= (_mm512_test_epi32_mask(v, open) | + _mm512_cmpeq_epi32_mask(_mm512_and_si512(v, unknown), unknown)) & 0x8888; + pg_fail |= (uint16_t)(~(_mm512_cmpeq_epi32_mask(_mm512_and_si512(v, mask), pg) | empty) & 0x1111); + sums = _mm512_mask_add_epi32(sums, 0x8888, sums, _mm512_srli_epi32(v, 24)); + } + *text_count = 64 - blanks; + *row_count = (uint32_t)_mm512_reduce_add_epi32(sums); + return (tag_fail | pg_fail) == 0; +} +#endif + +/* Drop whole lines in one pass. Update page references once per run. + * Use the normal path for open lines, continuations, and unknown row counts. */ +static uint64_t evict_simple_run(struct hist *h, uint64_t k) +{ + if (k == 0 || h->tail == h->head || !h->ring) + return 0; + const struct hline *ring = h->ring; + uint32_t rmask = h->ring_mask; + uint64_t tail = h->tail, head = h->head, base = h->base_id; + /* Neither side moves inside the loop, so count the lines once. */ + uint64_t counted = h->rows_id > base ? h->rows_id - base : 0; + uint64_t rows_take = 0; + uint64_t n = 0; + uint32_t page = 0, run = 0; + bool have_page = false; + + while (n < k && tail < head) { + if ((tail & 15) == 0 && n + 16 <= k && tail + 16 <= head && + (n >= counted || n + 16 <= counted)) { + const struct hist_block *b = &h->blocks[(tail & rmask) / 16]; + if (b->seq == tail + 1 && b->epoch == h->index_epoch && b->cols == h->wrap_cols) { + if (b->refs) { + if (have_page && page != b->page) { + page_drop_n(h, page, run); + run = 0; + } + page = b->page; + have_page = true; + run += b->refs; + } + if (n < counted) rows_take += b->rows; + tail += 16; + n += 16; + continue; + } + } +#if defined(TERM_KERNEL_AVX512) + if (term_use_avx512() && (tail & 63) == 0 && n + 64 <= k && tail + 64 <= head && + ((tail & rmask) + 64 <= rmask + 1)) { + const struct hline *f = &ring[tail & rmask]; + + uint32_t pg = f->off >> 16; + uint32_t text_count, row_count; + if (f->off != NO_TEXT && (n >= counted || n + 64 <= counted) && + same_page_rows_64(f, pg, &text_count, &row_count) && + (tail + 64 == head || !(ring[(tail + 64) & rmask].flags & HLINE_CONT))) { + if (have_page && page != pg) { + page_drop_n(h, page, run); + run = 0; + } + page = pg; + have_page = true; + run += text_count; + if (n < counted) rows_take += row_count; + tail += 64; + n += 64; + continue; + } + } +#endif + if ((tail & 15) == 0 && n + 16 <= k && tail + 16 <= head && ((tail & rmask) + 16 <= rmask + 1)) { + const struct hline *f = &ring[tail & rmask]; + + uint32_t pg = f->off >> 16; + if (f->off != NO_TEXT && same_page_narrow_rows(f, pg) && + (tail + 16 == head || !(ring[(tail + 16) & rmask].flags & HLINE_CONT))) { + if (have_page && page != pg) { + page_drop_n(h, page, run); + run = 0; + } + page = pg; + have_page = true; + run += 16; + uint64_t remaining = counted > n ? counted - n : 0; + rows_take += remaining < 16 ? remaining : 16; + tail += 16; + n += 16; + continue; + } + + uint32_t t0 = *(const uint32_t *)((const char *)&f[0] + 12); + uint32_t t1 = *(const uint32_t *)((const char *)&f[1] + 12); + uint32_t t2 = *(const uint32_t *)((const char *)&f[2] + 12); + uint32_t t3 = *(const uint32_t *)((const char *)&f[3] + 12); + uint32_t t4 = *(const uint32_t *)((const char *)&f[4] + 12); + uint32_t t5 = *(const uint32_t *)((const char *)&f[5] + 12); + uint32_t t6 = *(const uint32_t *)((const char *)&f[6] + 12); + uint32_t t7 = *(const uint32_t *)((const char *)&f[7] + 12); + uint32_t t8 = *(const uint32_t *)((const char *)&f[8] + 12); + uint32_t t9 = *(const uint32_t *)((const char *)&f[9] + 12); + uint32_t t10 = *(const uint32_t *)((const char *)&f[10] + 12); + uint32_t t11 = *(const uint32_t *)((const char *)&f[11] + 12); + uint32_t t12 = *(const uint32_t *)((const char *)&f[12] + 12); + uint32_t t13 = *(const uint32_t *)((const char *)&f[13] + 12); + uint32_t t14 = *(const uint32_t *)((const char *)&f[14] + 12); + uint32_t t15 = *(const uint32_t *)((const char *)&f[15] + 12); + + uint32_t or_all = t0 | t1 | t2 | t3 | t4 | t5 | t6 | t7 | + t8 | t9 | t10 | t11 | t12 | t13 | t14 | t15; + if (TERM_EXPECT((or_all & 0x0030ffff) == 0 && + (t0 >> 24) != 255 && (t1 >> 24) != 255 && (t2 >> 24) != 255 && (t3 >> 24) != 255 && + (t4 >> 24) != 255 && (t5 >> 24) != 255 && (t6 >> 24) != 255 && (t7 >> 24) != 255 && + (t8 >> 24) != 255 && (t9 >> 24) != 255 && (t10 >> 24) != 255 && (t11 >> 24) != 255 && + (t12 >> 24) != 255 && (t13 >> 24) != 255 && (t14 >> 24) != 255 && (t15 >> 24) != 255, 1)) { + + uint32_t o0 = f[0].off, o15 = f[15].off; + if (have_page && (o0 >> 16) == page && (o15 >> 16) == page && + o0 != NO_TEXT && f[1].off != NO_TEXT && f[2].off != NO_TEXT && f[3].off != NO_TEXT && + f[4].off != NO_TEXT && f[5].off != NO_TEXT && f[6].off != NO_TEXT && f[7].off != NO_TEXT && + f[8].off != NO_TEXT && f[9].off != NO_TEXT && f[10].off != NO_TEXT && f[11].off != NO_TEXT && + f[12].off != NO_TEXT && f[13].off != NO_TEXT && f[14].off != NO_TEXT && o15 != NO_TEXT) { + run += 16; + } else { + for (int j = 0; j < 16; j++) { + uint32_t o = f[j].off; + if (o != NO_TEXT) { + uint32_t pg = o >> 16; + if (have_page && pg == page) { + run++; + } else { + if (have_page) + page_drop_n(h, page, run); + page = pg; + run = 1; + have_page = true; + } + } + } + } + if (n + 16 <= counted) { + rows_take += (t0 >> 24) + (t1 >> 24) + (t2 >> 24) + (t3 >> 24) + + (t4 >> 24) + (t5 >> 24) + (t6 >> 24) + (t7 >> 24) + + (t8 >> 24) + (t9 >> 24) + (t10 >> 24) + (t11 >> 24) + + (t12 >> 24) + (t13 >> 24) + (t14 >> 24) + (t15 >> 24); + } + tail += 16; + n += 16; + continue; + } + } + + const struct hline *f = &ring[tail & rmask]; + if (f->flags & HLINE_OPEN) + break; + if (tail + 1 < head && (ring[(tail + 1) & rmask].flags & HLINE_CONT)) + break; + if (f->wrap_rows == ROWS_UNKNOWN) + break; + if (n < counted) + rows_take += f->wrap_rows; + if (f->off != NO_TEXT) { + uint32_t pg = f->off >> 16; + if (have_page && pg == page) { + run++; + } else { + if (have_page) + page_drop_n(h, page, run); + page = pg; + run = 1; + have_page = true; + } + } + tail++; + n++; + } + if (n == 0) + return 0; + if (have_page) + page_drop_n(h, page, run); + h->tail = tail; + h->base_id += n; + h->lines -= (uint32_t)n; + if (counted) { + h->counted_rows -= rows_take; + if (n >= counted) + h->rows_id = h->base_id; + } + return n; +} + +/* Make room for n more lines. Takes the simple lines in one pass, then + * falls back to one line at a time for the rest. The count is guarded + * before the subtraction: an unguarded `lines + n - line_limit` wraps when + * there is nothing to evict. */ +static inline void evict_for(struct hist *h, uint64_t n) +{ + uint64_t have = h->lines; + if (have + n <= (uint64_t)h->line_limit) + return; + uint64_t need = have + n - h->line_limit; + uint64_t done = evict_simple_run(h, need); + for (uint64_t i = done; i < need; i++) + evict_line(h); +} + +/* Drop the oldest line with all its parts. It is never the open line. */ +static inline TERM_ALWAYS_INLINE void evict_line(struct hist *h) +{ + if (h->tail == h->head) + return; + const struct hline *f = part(h, h->tail); + if (TERM_EXPECT(!(f->flags & HLINE_OPEN) && + (h->tail + 1 == h->head || !(part(h, h->tail + 1)->flags & HLINE_CONT)), 1)) { + if (h->base_id < h->rows_id) + h->counted_rows -= f->wrap_rows != ROWS_UNKNOWN + ? f->wrap_rows : term_hist_line_estimate(h, h->base_id, h->wrap_cols); + if (f->off != NO_TEXT) + page_drop_ref(h, f->off >> 16); + h->tail++; + line_gone(h); + return; + } + evict_line_parts(h); +} + +static TERM_NOINLINE int page_next(struct hist *h) +{ + for (;;) { + if (h->nfree) { + uint32_t pg = h->free_pages[--h->nfree]; + h->page_used[pg] = 0; + h->cur_page = pg; + return (int)pg; + } + if (h->npages < h->max_pages) { + uint8_t *page = term_aligned_alloc(64, HIST_PAGE_SIZE); + if (!page) + return -1; + uint32_t pg = h->npages++; + h->pages[pg] = page; + h->page_used[pg] = 0; + h->page_parts[pg] = 0; + h->cur_page = pg; + return (int)pg; + } + if (!evict_part(h)) + return -1; + } +} + +/* A page with need free bytes at its end. It becomes the current page. */ +static inline int page_for(struct hist *h, size_t need) +{ + /* No page can hold a request larger than the usable area. Returning a page + * here would let the caller copy past the end of the mapping. */ + if (need > HIST_PAGE_USABLE) + return -1; + uint32_t pg = h->cur_page; + if (TERM_EXPECT(pg < h->npages && h->page_used[pg] + need <= HIST_PAGE_USABLE, 1)) + return (int)pg; + return page_next(h); +} + +/* ---- Attribute runs ---- */ + +static inline int put_varint(uint8_t *p, uint32_t v) +{ + int n = 0; + while (v >= 0x80) { + p[n++] = (uint8_t)(v | 0x80); + v >>= 7; + } + p[n++] = (uint8_t)v; + return n; +} + +static inline int get_varint(const uint8_t *p, uint32_t *v) +{ + uint32_t r = 0; + int shift = 0, n = 0; + uint8_t b; + do { + b = p[n++]; + r |= (uint32_t)(b & 0x7f) << shift; + shift += 7; + } while ((b & 0x80) && shift < 35); + *v = r; + return n; +} + +static inline void put_u32(uint8_t *p, uint32_t v) { memcpy(p, &v, 4); } +static inline void put_u16(uint8_t *p, uint16_t v) { memcpy(p, &v, 2); } +static inline uint32_t get_u32(const uint8_t *p) { uint32_t v; memcpy(&v, p, 4); return v; } +static inline uint16_t get_u16(const uint8_t *p) { uint16_t v; memcpy(&v, p, 2); return v; } + +static inline uint8_t *emit_run(uint8_t *p, const struct pen_attr *base, const struct pen_attr *pen, + uint32_t count) +{ + /* Most runs change only colors and cover fewer than 128 cells. */ + if (TERM_EXPECT(count < 0x80 && pen_sl(pen) == pen_sl(base), 1)) { + uint32_t dfg = pen->fg != base->fg, dbg = pen->bg != base->bg; + p[0] = (uint8_t)(dfg * RUN_FG | dbg * RUN_BG); + p[1] = (uint8_t)count; + p += 2; + put_u32(p, pen->fg); + p += dfg * 4; + put_u32(p, pen->bg); + return p + dbg * 4; + } + uint8_t header = (pen->fg != base->fg ? RUN_FG : 0) | + (pen->bg != base->bg ? RUN_BG : 0) | + (pen->flags != base->flags ? RUN_ATTR : 0) | + (pen->link != base->link ? RUN_LINK : 0); + *p++ = header; + p += put_varint(p, count); + if (header & RUN_FG) { put_u32(p, pen->fg); p += 4; } + if (header & RUN_BG) { put_u32(p, pen->bg); p += 4; } + if (header & RUN_ATTR) { put_u16(p, pen->flags); p += 2; } + if (header & RUN_LINK) { put_u16(p, pen->link); p += 2; } + return p; +} + +static inline const uint8_t *read_run(const uint8_t *p, struct pen_attr *pen, uint32_t *count) +{ + uint8_t header = *p++; + p += get_varint(p, count); + if (header & RUN_FG) { pen->fg = get_u32(p); p += 4; } + if (header & RUN_BG) { pen->bg = get_u32(p); p += 4; } + if (header & RUN_ATTR) { pen->flags = get_u16(p); p += 2; } + if (header & RUN_LINK) { pen->link = get_u16(p); p += 2; } + return p; +} + +static bool runs_reserve(struct hist_runs *r, size_t extra) +{ + if (TERM_EXPECT(r->len + extra <= r->cap, 1)) + return true; + size_t cap = r->cap ? r->cap : 256; + while (cap < r->len + extra) + cap *= 2; + uint8_t *data = realloc(r->data, cap); + if (!data) + return false; + r->data = data; + r->cap = (uint32_t)cap; + return true; +} + +static inline void runs_reset(struct hist_runs *r) +{ + r->len = 0; + r->base = pen_attr_default(); + r->pen = pen_attr_default(); + r->count = 0; +} + +void term_hist_runs_emit_slow(struct hist_runs *r) +{ + r->len = (uint32_t)(emit_run(r->data + r->len, &r->base, &r->pen, r->count) - r->data); +} + +static inline void runs_repeat(struct hist_runs *r) +{ + hist_runs_switch(r, r->pen.fg, r->pen.bg, pen_sl(&r->pen)); +} + +/* Emit the pending run unless it is a default tail, which readers imply. */ +static void runs_finish(struct hist_runs *r) +{ + if (r->count && !pen_attr_is_default(&r->pen)) + runs_repeat(r); + r->count = 0; +} + +/* Emit the pending run, default or not, so the next run can use it as its + * delta base. */ +static void runs_flush(struct hist_runs *r) +{ + if (r->count) + runs_repeat(r); +} + +/* ---- Encoding ---- */ + +static inline size_t put_utf8(uint8_t *dst, uint32_t cp) +{ + if (cp < 0x80) { + dst[0] = (uint8_t)cp; + return 1; + } + if (cp < 0x800) { + put_u16(dst, (uint16_t)(0x80C0u | (cp >> 6) | ((cp & 0x3Fu) << 8))); + return 2; + } + if (cp < 0x10000) { + put_u32(dst, 0x008080E0u | (cp >> 12) | ((cp & 0x0FC0u) << 2) | ((cp & 0x3Fu) << 16)); + return 3; + } + if (cp <= 0x10FFFF) { + put_u32(dst, 0x808080F0u | (cp >> 18) | ((cp & 0x3F000u) >> 4) | + ((cp & 0x0FC0u) << 10) | ((cp & 0x3Fu) << 24)); + return 4; + } + put_u32(dst, 0x00BDBFEFu); + return 3; +} + +/* Decode one code point that this file wrote. It does not validate. */ +static inline int get_utf8(const uint8_t *s, const uint8_t *end, uint32_t *cp) +{ + uint8_t b = s[0]; + int n = b < 0xE0 ? (b < 0x80 ? 1 : 2) : b < 0xF0 ? 3 : 4; + if (s + n > end) { + *cp = 0xFFFD; + return (int)(end - s); + } + switch (n) { + case 1: *cp = b; break; + case 2: *cp = ((uint32_t)(b & 0x1f) << 6) | (s[1] & 0x3f); break; + case 3: *cp = ((uint32_t)(b & 0x0f) << 12) | ((uint32_t)(s[1] & 0x3f) << 6) | (s[2] & 0x3f); break; + default: + *cp = ((uint32_t)(b & 0x07) << 18) | ((uint32_t)(s[1] & 0x3f) << 12) | + ((uint32_t)(s[2] & 0x3f) << 6) | (s[3] & 0x3f); + break; + } + return n; +} + +struct enc { + uint32_t bytes; + uint32_t cells; + uint8_t flags; /* HAS_* flags; NARROW when every cell was one ASCII byte */ + uint16_t wide; +}; + +#if defined(TERM_KERNEL_AVX512) +/* The prefix of cells where (lane & keep) - target <= span holds in every + * lane. Their code points go to dst as bytes. Eight cells per step; the + * last step uses masked loads. */ +TERM_TARGET_AVX512 +static inline int narrow_prefix_avx512(const struct cell *cells, int len, uint8_t *dst, + __m512i keep, __m512i target, __m512i span) +{ + const __m512i idx = _mm512_setr_epi32(0, 4, 8, 12, 16, 20, 24, 28, 0, 0, 0, 0, 0, 0, 0, 0); + int k = 0; + for (; k < len; k += 8) { + int left = len - k; + __mmask16 m0 = left >= 4 ? 0xFFFF : (__mmask16)((1u << (left * 4)) - 1); + __mmask16 m1 = left >= 8 ? 0xFFFF : left <= 4 ? 0 : (__mmask16)((1u << ((left - 4) * 4)) - 1); + __m512i v0 = _mm512_maskz_loadu_epi32(m0, (const void *)(cells + k)); + __m512i v1 = _mm512_maskz_loadu_epi32(m1, (const void *)(cells + k + 4)); + __mmask16 ok0 = _mm512_mask_cmple_epu32_mask(m0, _mm512_sub_epi32(_mm512_and_si512(v0, keep), target), span); + __mmask16 ok1 = _mm512_mask_cmple_epu32_mask(m1, _mm512_sub_epi32(_mm512_and_si512(v1, keep), target), span); + __m512i cp = _mm512_permutex2var_epi32(v0, idx, v1); + if (ok0 != m0 || ok1 != m1) { + /* Keep the cells before the first failing one. */ + uint32_t bad = (uint32_t)(~ok0 & m0) | ((uint32_t)(~ok1 & m1) << 16); + int good = term_ctz32(bad) / 4; + _mm_mask_storeu_epi8(dst + k, (__mmask16)((1u << good) - 1), _mm512_cvtepi32_epi8(cp)); + return k + good; + } + if (left >= 8) + _mm_storel_epi64((__m128i *)(void *)(dst + k), _mm512_cvtepi32_epi8(cp)); + else + _mm_mask_storeu_epi8(dst + k, (__mmask16)((1u << left) - 1), _mm512_cvtepi32_epi8(cp)); + } + return len; +} + +TERM_TARGET_AVX512 +static inline int narrow_any_pen_prefix_avx512(const struct cell *cells, int len, uint8_t *dst) +{ + const __m512i keep = _mm512_setr_epi32(-1, 0, 0, SHAPE_FLAGS, -1, 0, 0, SHAPE_FLAGS, + -1, 0, 0, SHAPE_FLAGS, -1, 0, 0, SHAPE_FLAGS); + const __m512i target = _mm512_setr_epi32(0x20, 0, 0, 0, 0x20, 0, 0, 0, 0x20, 0, 0, 0, 0x20, 0, 0, 0); + const __m512i span = _mm512_setr_epi32(0x5f, 0, 0, 0, 0x5f, 0, 0, 0, 0x5f, 0, 0, 0, 0x5f, 0, 0, 0); + return narrow_prefix_avx512(cells, len, dst, keep, target, span); +} + +TERM_TARGET_AVX512 +static inline int narrow_uniform_prefix_avx512(const struct cell *cells, int len, uint8_t *dst) +{ + uint32_t sl = (uint32_t)(cells[0].flags & STYLE_FLAGS) | ((uint32_t)cells[0].pad << 16); + const __m512i target = _mm512_setr_epi32( + 0x20, (int)cells[0].fg, (int)cells[0].bg, (int)sl, 0x20, (int)cells[0].fg, (int)cells[0].bg, (int)sl, + 0x20, (int)cells[0].fg, (int)cells[0].bg, (int)sl, 0x20, (int)cells[0].fg, (int)cells[0].bg, (int)sl); + const __m512i keep = _mm512_setr_epi32( + -1, -1, -1, (int)(STYLE_FLAGS | SHAPE_FLAGS | 0xFFFF0000u), -1, -1, -1, (int)(STYLE_FLAGS | SHAPE_FLAGS | 0xFFFF0000u), + -1, -1, -1, (int)(STYLE_FLAGS | SHAPE_FLAGS | 0xFFFF0000u), -1, -1, -1, (int)(STYLE_FLAGS | SHAPE_FLAGS | 0xFFFF0000u)); + const __m512i span = _mm512_setr_epi32(0x5f, 0, 0, 0, 0x5f, 0, 0, 0, 0x5f, 0, 0, 0, 0x5f, 0, 0, 0); + return narrow_prefix_avx512(cells, len, dst, keep, target, span); +} +#endif + +/* The prefix of cells that are printable ASCII with any pen. Their bytes go + * to dst. */ +static inline int narrow_any_pen_prefix(const struct cell *cells, int len, uint8_t *dst) +{ +#if defined(TERM_KERNEL_AVX512) + if (term_use_avx512()) + return narrow_any_pen_prefix_avx512(cells, len, dst); +#endif + int k = 0; + for (; k < len; k++) { + if (cells[k].c - 0x20u >= 0x60u || (cells[k].flags & SHAPE_FLAGS)) + break; + dst[k] = (uint8_t)cells[k].c; + } + return k; +} + +/* Add the pens of n cells to r, one run per change. Cells carry no + * underline color, so no pen here has one. */ +static TERM_NOINLINE uint32_t runs_scan(struct hist_runs *r, const struct cell *cells, int n) +{ + uint32_t links = 0; + uint32_t pfg = r->pen.fg, pbg = r->pen.bg; + uint32_t psl = (uint32_t)r->pen.flags | ((uint32_t)r->pen.link << 16); + uint32_t bfg = r->base.fg, bbg = r->base.bg; + uint32_t bsl = (uint32_t)r->base.flags | ((uint32_t)r->base.link << 16); + uint32_t count = r->count; + uint8_t *rp = r->data + r->len; + for (int i = 0; i < n; i++) { + const struct cell *c = &cells[i]; + uint32_t sl = (uint32_t)(c->flags & STYLE_FLAGS) | ((uint32_t)c->pad << 16); + links |= c->pad; + if (TERM_EXPECT(((c->fg ^ pfg) | (c->bg ^ pbg) | (sl ^ psl)) != 0, 0)) { + if (count) { + uint32_t header = (pfg != bfg ? RUN_FG : 0) | (pbg != bbg ? RUN_BG : 0) | + ((uint16_t)psl != (uint16_t)bsl ? RUN_ATTR : 0) | + ((psl >> 16) != (bsl >> 16) ? RUN_LINK : 0); + rp[0] = (uint8_t)header; + if (TERM_EXPECT(count < 0x80, 1)) { + rp[1] = (uint8_t)count; + rp += 2; + } else { + rp += 1 + put_varint(rp + 1, count); + } + put_u32(rp, pfg); + rp += (header & RUN_FG) ? 4 : 0; + put_u32(rp, pbg); + rp += (header & RUN_BG) ? 4 : 0; + if (TERM_EXPECT(header & (RUN_ATTR | RUN_LINK), 0)) { + if (header & RUN_ATTR) { put_u16(rp, (uint16_t)psl); rp += 2; } + if (header & RUN_LINK) { put_u16(rp, (uint16_t)(psl >> 16)); rp += 2; } + } + bfg = pfg; + bbg = pbg; + bsl = psl; + } + pfg = c->fg; + pbg = c->bg; + psl = sl; + count = 0; + } + count++; + } + r->len = (uint32_t)(rp - r->data); + pen_set(&r->base, bfg, bbg, bsl); + pen_set(&r->pen, pfg, pbg, psl); + r->count = count; + return links; +} + +/* The prefix of cells that are printable ASCII with the pen of cells[0]. + * Their bytes go to dst. */ +static inline int narrow_uniform_prefix(const struct cell *cells, int len, uint8_t *dst) +{ +#if defined(TERM_KERNEL_AVX512) + if (term_use_avx512()) + return narrow_uniform_prefix_avx512(cells, len, dst); +#endif + uint32_t sl = (uint32_t)(cells[0].flags & STYLE_FLAGS) | ((uint32_t)cells[0].pad << 16); + int k = 0; + for (; k < len; k++) { + const struct cell *c = &cells[k]; + uint32_t csl = (uint32_t)(c->flags & (STYLE_FLAGS | SHAPE_FLAGS)) | ((uint32_t)c->pad << 16); + if (c->c - 0x20u >= 0x60u || c->fg != cells[0].fg || c->bg != cells[0].bg || csl != sl) + break; + dst[k] = (uint8_t)c->c; + } + return k; +} + +#if defined(TERM_KERNEL_AVX512) +/* Encode a prefix of cells with the pen of cells[0] and no clusters, eight + * cells at a time. Return the cells used; runcells counts the cells that + * are not trail cells. */ +TERM_TARGET_AVX512 +static int uniform_utf8_prefix(const struct cell *cells, int len, uint8_t *dst, struct enc *o, + uint32_t *runcells) +{ + uint32_t sl = (uint32_t)(cells[0].flags & STYLE_FLAGS) | ((uint32_t)cells[0].pad << 16); + const __m512i target = _mm512_setr_epi32( + 0, (int)cells[0].fg, (int)cells[0].bg, (int)sl, 0, (int)cells[0].fg, (int)cells[0].bg, (int)sl, + 0, (int)cells[0].fg, (int)cells[0].bg, (int)sl, 0, (int)cells[0].fg, (int)cells[0].bg, (int)sl); + /* The code point lane only checks that the value is below 0x10000. The + * flags lane allows WIDE and TRAIL. */ + const __m512i keep = _mm512_setr_epi32( + (int)0xFFFF0000u, -1, -1, (int)(STYLE_FLAGS | CELL_WRAP_PAD | CELL_EMOJI | CELL_TEXT | 0xFFFF0000u), + (int)0xFFFF0000u, -1, -1, (int)(STYLE_FLAGS | CELL_WRAP_PAD | CELL_EMOJI | CELL_TEXT | 0xFFFF0000u), + (int)0xFFFF0000u, -1, -1, (int)(STYLE_FLAGS | CELL_WRAP_PAD | CELL_EMOJI | CELL_TEXT | 0xFFFF0000u), + (int)0xFFFF0000u, -1, -1, (int)(STYLE_FLAGS | CELL_WRAP_PAD | CELL_EMOJI | CELL_TEXT | 0xFFFF0000u)); + const __m512i idx_cp = _mm512_setr_epi32(0, 4, 8, 12, 16, 20, 24, 28, 0, 0, 0, 0, 0, 0, 0, 0); + const __m512i idx_fl = _mm512_setr_epi32(3, 7, 11, 15, 19, 23, 27, 31, 0, 0, 0, 0, 0, 0, 0, 0); + int i = 0; + uint32_t bytes = 0, ncells = 0, wide = 0, ascii = 0xFF, nwide = 0; + for (; i + 8 <= len; i += 8) { + __m512i v0 = _mm512_loadu_si512((const void *)(cells + i)); + __m512i v1 = _mm512_loadu_si512((const void *)(cells + i + 4)); + __m512i d = _mm512_or_si512(_mm512_xor_si512(_mm512_and_si512(v0, keep), target), + _mm512_xor_si512(_mm512_and_si512(v1, keep), target)); + if (_mm512_test_epi32_mask(d, d)) + break; + __m256i cp = _mm512_castsi512_si256(_mm512_permutex2var_epi32(v0, idx_cp, v1)); + __m256i fl = _mm512_castsi512_si256(_mm512_permutex2var_epi32(v0, idx_fl, v1)); + __m256i ch = _mm256_max_epu32(cp, _mm256_set1_epi32(0x20)); + __mmask8 ge80 = _mm256_cmpge_epu32_mask(ch, _mm256_set1_epi32(0x80)); + __mmask8 ge800 = _mm256_cmpge_epu32_mask(ch, _mm256_set1_epi32(0x800)); + __mmask8 m_wide = _mm256_test_epi32_mask(fl, _mm256_set1_epi32(CELL_WIDE)); + __mmask8 m_trail = _mm256_test_epi32_mask(fl, _mm256_set1_epi32(CELL_TRAIL)); + __mmask8 surrogate = _mm256_cmplt_epu32_mask(_mm256_sub_epi32(ch, _mm256_set1_epi32(0xD800)), + _mm256_set1_epi32(0x800)); + if ((m_wide & ~ge800) || (surrogate & ~m_trail)) + break; + /* Lane bytes, low first: F5 E? 8? 8?. A wide lane keeps all four, a + * three-byte lane the last three. A two-byte lane turns byte 2 into + * its lead byte (C0 | cp >> 6) and keeps the last two. ASCII keeps + * byte 3 only. */ + __m256i b1 = _mm256_and_si256(_mm256_srli_epi32(ch, 4), _mm256_set1_epi32(0x00000F00)); + __m256i b2 = _mm256_and_si256(_mm256_slli_epi32(ch, 10), _mm256_set1_epi32(0x003F0000)); + __m256i b3 = _mm256_and_si256(_mm256_slli_epi32(ch, 24), _mm256_set1_epi32(0x3F000000)); + __m256i w = _mm256_or_si256(_mm256_or_si256(b1, b2), _mm256_or_si256(b3, _mm256_set1_epi32((int)0x8080E0F5u))); + w = _mm256_mask_or_epi32(w, (__mmask8)~ge800, w, _mm256_set1_epi32(0x00400000)); + w = _mm256_mask_mov_epi32(w, (__mmask8)~ge80, _mm256_slli_epi32(ch, 24)); + __m256i bm = _mm256_set1_epi32((int)0x80000000u); + bm = _mm256_mask_mov_epi32(bm, ge80, _mm256_set1_epi32((int)0x80800000u)); + bm = _mm256_mask_mov_epi32(bm, ge800, _mm256_set1_epi32((int)0x80808000u)); + bm = _mm256_mask_mov_epi32(bm, m_wide, _mm256_set1_epi32((int)0x80808080u)); + bm = _mm256_maskz_mov_epi32((__mmask8)~m_trail, bm); + __mmask32 keep_bytes = _mm256_movepi8_mask(bm); + _mm256_storeu_si256((__m256i *)(void *)(dst + bytes), _mm256_maskz_compress_epi8(keep_bytes, w)); + bytes += (uint32_t)term_popcount32(keep_bytes); + uint32_t trails = (uint32_t)term_popcount32(m_trail); + ncells += 8u - trails + (uint32_t)term_popcount32(m_wide); + *runcells += 8u - trails; + nwide += (uint32_t)term_popcount32(m_wide); + wide |= m_wide; + ascii &= (uint32_t)(uint8_t)~(ge80 & ~m_trail); + } + o->bytes = bytes; + o->cells = ncells; + o->wide = (uint16_t)nwide; + o->flags = (wide ? HLINE_HAS_WIDE : 0) | (ascii == 0xFF ? HLINE_NARROW : 0) | + (cells[0].pad ? HLINE_HAS_LINK : 0); + return i; +} +#endif + +static TERM_NOINLINE size_t encode_cluster(uint8_t *dst, const struct gstore *g, + uint32_t handle, uint16_t f) +{ + uint32_t cps[GRAPHEME_MAX_LEN]; + int n = term_gstore_expand(g, handle, cps, GRAPHEME_MAX_LEN); + size_t b = 0; + dst[b++] = PREFIX_CLUSTER; + dst[b++] = (uint8_t)((n & 0x1f) | ((f & CELL_WIDE) ? 0x80 : 0) | + ((f & CELL_EMOJI) ? 0x40 : 0) | ((f & CELL_TEXT) ? 0x20 : 0)); + for (int k = 0; k < n; k++) + b += put_utf8(dst + b, cps[k]); + return b; +} + +static size_t encode_shaped(uint8_t *dst, uint32_t cp, uint16_t f, const struct gstore *g, + uint8_t *flags) +{ + if (grapheme_is_handle(cp)) { + *flags |= HLINE_HAS_CLUSTER | ((f & CELL_WIDE) ? HLINE_HAS_WIDE : 0); + return encode_cluster(dst, g, cp, f); + } + if (cp < 0x20) + cp = ' '; + uint16_t shape = f & (CELL_WIDE | CELL_EMOJI | CELL_TEXT); + size_t b = 0; + if (shape == 0) + return put_utf8(dst, cp); + if (shape == CELL_WIDE) { + *flags |= HLINE_HAS_WIDE; + dst[b++] = PREFIX_WIDE; + } else if (shape == (CELL_WIDE | CELL_EMOJI)) { + *flags |= HLINE_HAS_WIDE; + dst[b++] = PREFIX_EMOJI; + } else if (shape == CELL_TEXT) { + dst[b++] = PREFIX_TEXT; + } else { + *flags |= HLINE_HAS_CLUSTER | ((f & CELL_WIDE) ? HLINE_HAS_WIDE : 0); + dst[b++] = PREFIX_CLUSTER; + dst[b++] = (uint8_t)(1 | ((f & CELL_WIDE) ? 0x80 : 0) | + ((f & CELL_EMOJI) ? 0x40 : 0) | ((f & CELL_TEXT) ? 0x20 : 0)); + } + return b + put_utf8(dst + b, cp); +} + +/* Encode cells[0..n) into dst and add their pens to r. Stop before a + * cluster that could overflow cap. Return the cells used. The caller + * reserved (n + 1) * RUN_MAX bytes in r. */ +static int encode_cells(struct hist_runs *r, const struct cell *cells, int n, uint8_t *dst, + size_t cap, const struct gstore *g, struct enc *o) +{ + o->bytes = 0; + o->cells = 0; + o->flags = HLINE_NARROW; + o->wide = 0; + int i = 0; + if (n <= 0) + return 0; + const struct cell *c0 = &cells[0]; + struct pen_attr pen0 = { c0->fg, c0->bg, (uint16_t)(c0->flags & STYLE_FLAGS), c0->pad }; + bool ascii0 = !(c0->flags & SHAPE_FLAGS) && c0->c - 0x20u < 0x60u; + if (ascii0) + i = narrow_uniform_prefix(cells, n, dst); + if (i == n) { + hist_runs_add(r, &pen0, (uint32_t)n); + *o = (struct enc){ .bytes = (uint32_t)n, .cells = (uint32_t)n, + .flags = (uint8_t)(HLINE_NARROW | (c0->pad ? HLINE_HAS_LINK : 0)) }; + return n; + } + const struct cell *ci = &cells[i]; + if (i > 0 && !(ci->flags & SHAPE_FLAGS) && ci->c - 0x20u < 0x60u) { + /* The pen changed inside ASCII: a pass for the bytes, one for the pens. */ + i += narrow_any_pen_prefix(ci, n - i, dst + i); + uint32_t links = runs_scan(r, cells, i); + *o = (struct enc){ .bytes = (uint32_t)i, .cells = (uint32_t)i, + .flags = (uint8_t)(HLINE_NARROW | (links ? HLINE_HAS_LINK : 0)) }; + if (i == n) + return n; + } else if (i > 0) { + hist_runs_add(r, &pen0, (uint32_t)i); + *o = (struct enc){ .bytes = (uint32_t)i, .cells = (uint32_t)i, + .flags = (uint8_t)(HLINE_NARROW | (c0->pad ? HLINE_HAS_LINK : 0)) }; + } +#if defined(TERM_KERNEL_AVX512) + if (term_use_avx512() && n - i >= 8 && ((cells[i].flags & SHAPE_FLAGS) || cells[i].c >= 0x80)) { + /* Text with one pen, eight cells per step. */ + struct enc u; + uint32_t runcells = 0; + const struct cell *cu = &cells[i]; + int k = uniform_utf8_prefix(cu, n - i, dst + o->bytes, &u, &runcells); + if (k > 0) { + hist_runs_add_pen(r, cu->fg, cu->bg, + (uint32_t)(cu->flags & STYLE_FLAGS) | ((uint32_t)cu->pad << 16), runcells); + o->bytes += u.bytes; + o->cells += u.cells; + o->wide += u.wide; + o->flags = (uint8_t)((o->flags & u.flags & HLINE_NARROW) | + ((o->flags | u.flags) & ~HLINE_NARROW)); + i += k; + if (i == n) + return n; + } + } +#endif + + /* The pending run and the delta base stay in registers. */ + uint32_t pfg = r->pen.fg, pbg = r->pen.bg, psl = pen_sl(&r->pen); + uint32_t bfg = r->base.fg, bbg = r->base.bg, bsl = pen_sl(&r->base); + uint32_t count = r->count; + uint8_t *rp = r->data + r->len; + uint32_t bytes = o->bytes, ncells = o->cells, nwide = o->wide; + uint8_t flags = o->flags; + uint32_t links = 0; + for (; i < n; i++) { + const struct cell *c = &cells[i]; + uint16_t f = c->flags; + if (f & (CELL_TRAIL | CELL_WRAP_PAD)) + continue; + uint32_t cp = c->c; + bool simple = !(f & (CELL_WIDE | CELL_EMOJI | CELL_TEXT)) && cp - 0x20u < 0x60u; + if (!simple && grapheme_is_handle(cp) && + bytes + CLUSTER_BYTES + (size_t)(n - i) * CELL_BYTES > cap) + break; + uint32_t sl = (uint32_t)(f & STYLE_FLAGS) | ((uint32_t)c->pad << 16); + links |= c->pad; + if (TERM_EXPECT(((c->fg ^ pfg) | (c->bg ^ pbg) | (sl ^ psl)) != 0, 0)) { + if (count) { + uint32_t header = (pfg != bfg ? RUN_FG : 0) | (pbg != bbg ? RUN_BG : 0) | + ((uint16_t)psl != (uint16_t)bsl ? RUN_ATTR : 0) | + ((psl >> 16) != (bsl >> 16) ? RUN_LINK : 0); + *rp++ = (uint8_t)header; + if (TERM_EXPECT(count < 0x80, 1)) + *rp++ = (uint8_t)count; + else + rp += put_varint(rp, count); + put_u32(rp, pfg); + rp += (header & RUN_FG) ? 4 : 0; + put_u32(rp, pbg); + rp += (header & RUN_BG) ? 4 : 0; + if (TERM_EXPECT(header & (RUN_ATTR | RUN_LINK), 0)) { + if (header & RUN_ATTR) { put_u16(rp, (uint16_t)psl); rp += 2; } + if (header & RUN_LINK) { put_u16(rp, (uint16_t)(psl >> 16)); rp += 2; } + } + bfg = pfg; + bbg = pbg; + bsl = psl; + } + pfg = c->fg; + pbg = c->bg; + psl = sl; + count = 0; + } + count++; + nwide += (f & CELL_WIDE) != 0; + if (TERM_EXPECT(simple, 1)) { + dst[bytes++] = (uint8_t)cp; + ncells++; + continue; + } + flags &= (uint8_t)~HLINE_NARROW; + uint16_t shape = f & (CELL_WIDE | CELL_EMOJI | CELL_TEXT); + if ((shape == 0 || shape == CELL_WIDE) && cp - 0x80u < 0xD800u - 0x80u) { + /* Narrow or wide text below the surrogates: inline. */ + if (shape) { + dst[bytes++] = PREFIX_WIDE; + flags |= HLINE_HAS_WIDE; + ncells++; + } + bytes += (uint32_t)put_utf8(dst + bytes, cp); + ncells++; + continue; + } + bytes += (uint32_t)encode_shaped(dst + bytes, cp, f, g, &flags); + ncells += (f & CELL_WIDE) ? 2 : 1; + } + r->len = (uint32_t)(rp - r->data); + pen_set(&r->base, bfg, bbg, bsl); + pen_set(&r->pen, pfg, pbg, psl); + r->count = count; + o->bytes = bytes; + o->cells = ncells; + o->wide = (uint16_t)nwide; + o->flags = flags | (links ? HLINE_HAS_LINK : 0); + return i; +} + +/* ---- Writing ---- */ + +static inline uint8_t row_count_byte(uint64_t rows) +{ + return rows < ROWS_UNKNOWN ? (uint8_t)rows : ROWS_UNKNOWN; +} + +/* + * The fixed half of a narrow descriptor: wide 0, attr_len 0, + * HLINE_NARROW, and wrap_rows 1. Only `cells` varies inside it, so a + * one-row narrow line shares one 64-bit constant. Byte 8 is cells, + * byte 14 is flags, and byte 15 is wrap_rows. + */ +#define HLINE_NARROW_TAIL ((uint64_t)HLINE_NARROW << 48 | 1ULL << 56) + +/* Every blank part is the same 16 bytes: no text, no cells, one row. + * Reading the constant beats zeroing 16 bytes and patching fields. */ +static const struct hline HLINE_BLANK = { + .off = NO_TEXT, .flags = HLINE_NARROW, .wrap_rows = 1, +}; + +static inline void part_put_narrow_rows(struct hline *p, uint32_t off, uint32_t len, uint32_t rows) +{ + uint64_t v[2] = { + (uint64_t)off | (uint64_t)len << 32, + (uint64_t)len | ((uint64_t)HLINE_NARROW << 48) | ((uint64_t)row_count_byte(rows) << 56), + }; + memcpy(p, v, sizeof(v)); +} + +/* Store the descriptor of one narrow line that holds a single row and uses + * the default pen. This is the common case on the write path. The whole + * 16 bytes leave in one store. The compound literal form instead costs a + * 16-byte zero store plus four field patches. */ +static inline void part_put_narrow(struct hline *p, uint32_t off, uint32_t len) +{ + uint64_t v[2] = { + (uint64_t)off | (uint64_t)len << 32, + (uint64_t)len | HLINE_NARROW_TAIL, + }; + memcpy(p, v, sizeof(v)); +} + +static void walk_advance(struct hist *h); + +/* Descriptor estimate. A viewport read accounts for wide wrap pads. */ +static inline uint64_t cheap_rows(uint64_t cells, uint64_t wide, int cols) +{ + return rows_narrow(cols == 1 ? cells - wide : cells, cols); +} + +/* Count the last line, which just closed, when it is next in order. */ +static inline void count_closed(struct hist *h, struct hline *first, uint64_t rows) +{ + if (rows && h->wrap_cols && h->rows_id == hist_end_id(h) - 1) { + first->wrap_rows = row_count_byte(rows); + h->counted_rows += rows; + h->rows_id++; + } +} + +/* A whole line in one push: text, runs, and one descriptor. It returns false + * when a cluster does not fit; nothing is committed then. */ +static bool push_line(struct hist *h, const struct cell *cells, int len, const char *ascii, + const struct pen_attr *pen, const struct gstore *g) +{ + if (h->lines >= h->line_limit) + evict_line(h); + size_t runs_need = ascii ? RUN_MAX * 2 : (size_t)(len + 2) * RUN_MAX; + size_t text_need = ascii ? (size_t)len : (size_t)len * CELL_BYTES; + int pg = page_for(h, text_need + runs_need + RUN_MAX); + if (pg < 0 || !ring_reserve(h) || !runs_reserve(&h->runs, runs_need + RUN_MAX)) + return false; + uint32_t off = h->page_used[pg]; + uint8_t *dst = h->pages[pg] + off; + runs_reset(&h->runs); + struct enc e; + if (ascii) { + bytes_copy_small(dst, ascii, len); + hist_runs_add(&h->runs, pen, (uint32_t)len); + e = (struct enc){ .bytes = (uint32_t)len, .cells = (uint32_t)len, + .flags = (uint8_t)(HLINE_NARROW | (pen->link ? HLINE_HAS_LINK : 0)) }; + } else { + size_t cap = HIST_PAGE_USABLE - off - (runs_need + RUN_MAX); + if (encode_cells(&h->runs, cells, len, dst, cap, g, &e) < len) + return false; + } + runs_finish(&h->runs); + if (h->runs.len) + memcpy(dst + e.bytes, h->runs.data, h->runs.len); + struct hline *p = part(h, h->head++); + *p = (struct hline){ + .off = ((uint32_t)pg << 16) | off, + .text_len = e.bytes, + .cells = (uint16_t)e.cells, + .wide = e.wide, + .attr_len = (uint16_t)h->runs.len, + .flags = e.flags, + .wrap_rows = ROWS_UNKNOWN, + }; + h->page_used[pg] = off + e.bytes + h->runs.len; + h->page_parts[pg]++; + h->lines++; + h->all_blank = false; + h->runs.len = 0; + if (h->wrap_cols) + count_closed(h, p, cheap_rows(e.cells, e.wide, h->wrap_cols)); + return true; +} + +/* A whole line of printable ASCII cells with one pen: the bytes, at most one + * run, one descriptor. Return false, with nothing committed, for any other + * line. */ +static inline bool push_uniform_line(struct hist *h, const struct cell *cells, int len) +{ + if (h->lines >= h->line_limit) + evict_line(h); + uint32_t pg = h->cur_page; + if (pg >= h->npages || h->page_used[pg] + (uint32_t)len + RUN_MAX > HIST_PAGE_USABLE || + !h->ring || h->head - h->tail > h->ring_mask) + return false; + uint32_t off = h->page_used[pg]; + uint8_t *dst = h->pages[pg] + off; + if (narrow_uniform_prefix(cells, len, dst) != len) + return false; + struct pen_attr pen = { cells[0].fg, cells[0].bg, (uint16_t)(cells[0].flags & STYLE_FLAGS), + cells[0].pad }; + uint32_t attr = 0; + if (!pen_attr_is_default(&pen)) { + struct pen_attr base = pen_attr_default(); + attr = (uint32_t)(emit_run(dst + len, &base, &pen, (uint32_t)len) - (dst + len)); + } + struct hline *q = part(h, h->head++); + *q = (struct hline){ + .off = (pg << 16) | off, .text_len = (uint32_t)len, .cells = (uint32_t)len, + .attr_len = (uint16_t)attr, + .flags = (uint8_t)(HLINE_NARROW | (pen.link ? HLINE_HAS_LINK : 0)), + .wrap_rows = ROWS_UNKNOWN, + }; + h->page_used[pg] = off + (uint32_t)len + attr; + h->page_parts[pg]++; + h->lines++; + h->all_blank = false; + if (h->wrap_cols) + count_closed(h, q, cheap_rows((uint64_t)len, 0, h->wrap_cols)); + return true; +} + +/* Start a line. The caller made sure that no line is open. */ +static struct hline *begin_line(struct hist *h, size_t need) +{ + if (h->lines >= h->line_limit) + evict_line(h); + int pg = page_for(h, need); + if (pg < 0 || !ring_reserve(h)) + return NULL; + struct hline *p = part(h, h->head++); + *p = (struct hline){ + .off = ((uint32_t)pg << 16) | h->page_used[pg], + .flags = HLINE_NARROW | HLINE_OPEN, + .wrap_rows = ROWS_UNKNOWN, + }; + h->page_parts[pg]++; + h->lines++; + h->last_id = hist_end_id(h) - 1; + h->last_cells = h->last_wide = 0; + h->all_blank = false; + h->walk_on = false; + runs_reset(&h->runs); + return p; +} + +/* Write the runs of the open part after its text. */ +static void close_part(struct hist *h, struct hline *p) +{ + runs_finish(&h->runs); + uint32_t pg = p->off >> 16; + uint32_t end = (p->off & 0xffff) + p->text_len; + if (h->runs.len) { + memcpy(h->pages[pg] + end, h->runs.data, h->runs.len); + p->attr_len = (uint16_t)h->runs.len; + end += h->runs.len; + } + h->page_used[pg] = end; + p->flags &= (uint8_t)~HLINE_OPEN; +} + +void term_hist_close_open_line(struct hist *h) +{ + struct hline *p = open_part(h); + if (!p) + return; + close_part(h, p); + runs_reset(&h->runs); + uint64_t id = hist_end_id(h) - 1; + int cols = h->wrap_cols; + if (!cols) { + h->walk_on = false; + return; + } + uint64_t first = line_first(h, id); + uint64_t rows = first == h->head - 1 + ? cheap_rows(p->cells, p->wide, cols) + : cheap_rows(term_hist_line_cells(h, id), term_hist_line_wide(h, id), cols); + if (!rows && h->walk_on) { + walk_advance(h); + rows = h->walk_rows; + } + h->walk_on = false; + count_closed(h, part(h, first), rows); +} + +/* Make room in the open part for text_need bytes and runs_need run bytes. + * Relocate a small part or continue the line in a new part. */ +static struct hline *open_room(struct hist *h, struct hline *p, size_t text_need, + size_t runs_need, uint32_t add_cells) +{ + uint32_t start = p->off & 0xffff; + size_t runs = h->runs.len + runs_need + RUN_MAX; + if (TERM_EXPECT(start + p->text_len + text_need + runs <= HIST_PAGE_USABLE && + p->cells + add_cells <= HIST_MAX_PART_CELLS && + runs <= UINT16_MAX, 1)) + return p; + + if (p->text_len + text_need + runs <= HIST_PAGE_USABLE / 4 && + p->cells + add_cells <= HIST_MAX_PART_CELLS && start > 0) { + uint32_t old_pg = p->off >> 16; + int pg = h->page_parts[old_pg] == 1 ? -1 : page_next(h); + if (pg >= 0) { + memcpy(h->pages[pg], h->pages[old_pg] + start, p->text_len); + if (h->page_used[old_pg] == start + p->text_len) + h->page_used[old_pg] = start; + page_drop_ref(h, old_pg); + p->off = (uint32_t)pg << 16; + h->page_parts[pg]++; + h->page_used[pg] = p->text_len; + return p; + } + /* The part is alone on its page, or no other page is free and the + * evictions left it alone: move it to the page start. */ + if (h->page_parts[old_pg] == 1) { + memmove(h->pages[old_pg], h->pages[old_pg] + start, p->text_len); + p->off = old_pg << 16; + if (h->cur_page != old_pg && h->cur_page < h->npages && !h->page_parts[h->cur_page]) + h->free_pages[h->nfree++] = (uint16_t)h->cur_page; + h->cur_page = old_pg; + h->page_used[old_pg] = p->text_len; + return p; + } + } + + /* Continue the line in a new part. The old part stays OPEN until the new + * part exists, so a page eviction does not drop the line. */ + struct pen_attr pen = h->runs.pen; + runs_finish(&h->runs); + uint32_t pg0 = p->off >> 16; + uint32_t end = (p->off & 0xffff) + p->text_len; + if (h->runs.len) { + memcpy(h->pages[pg0] + end, h->runs.data, h->runs.len); + p->attr_len = (uint16_t)h->runs.len; + end += h->runs.len; + } + h->page_used[pg0] = end; + int pg = page_next(h); + if (pg < 0) { + /* Only the open part is left and no page is free. Drop its content + * and reuse its page. The line keeps its id. */ + if (h->tail + 1 != h->head || h->page_parts[pg0] != 1) + return NULL; + h->page_used[pg0] = 0; + if (h->base_cell_id != h->base_id) { + h->base_cell_id = h->base_id; + h->base_cell = 0; + } + h->base_cell += p->cells; + h->last_cells = h->last_wide = 0; + h->index_epoch++; + h->cur_page = pg0; + *p = (struct hline){ + .off = pg0 << 16, + .flags = HLINE_OPEN | HLINE_NARROW | (p->flags & HLINE_CONT), + .wrap_rows = ROWS_UNKNOWN, + }; + h->walk_on = false; + runs_reset(&h->runs); + h->runs.pen = pen; + return p; + } + if (!ring_reserve(h)) + return NULL; + p = part(h, h->head - 1); + p->flags &= (uint8_t)~HLINE_OPEN; + struct hline *q = part(h, h->head++); + *q = (struct hline){ + .off = ((uint32_t)pg << 16) | h->page_used[pg], + .flags = HLINE_OPEN | HLINE_CONT | HLINE_NARROW, + .wrap_rows = ROWS_UNKNOWN, + }; + h->page_parts[pg]++; + if (!cont_push(h, h->head - 1)) { + /* Without the index entry, line_first() would map the next logical + * line to this part and scrollback would show the wrong text. Put + * the line back the way it was: the old part is open again and the + * new part never existed. */ + h->page_parts[pg]--; + h->head--; + p->flags |= HLINE_OPEN; + return NULL; + } + runs_reset(&h->runs); + h->runs.pen = pen; + return q; +} + +static inline void part_add(struct hist *h, struct hline *p, const struct enc *e) +{ + p->text_len += e->bytes; + p->cells += (uint16_t)e->cells; + p->wide += e->wide; + h->last_cells += e->cells; + h->last_wide += e->wide; + p->flags = (uint8_t)((p->flags & ~HLINE_NARROW) | (e->flags & ~HLINE_NARROW) | + (p->flags & e->flags & HLINE_NARROW)); + h->page_used[p->off >> 16] = (p->off & 0xffff) + p->text_len; +} + +void term_hist_push_row(struct hist *h, const struct cell *cells, int len, bool wrapped, + const struct gstore *g) +{ + if (!h->line_limit) + return; + struct hline *p = open_part(h); + if (!p) { + if (len <= 0 && !wrapped) { + term_hist_push_blank(h, 1); + return; + } + if (!wrapped && push_uniform_line(h, cells, len)) + return; + if (!wrapped && push_line(h, cells, len, NULL, NULL, g)) + return; + p = begin_line(h, (size_t)len * CELL_BYTES + (size_t)(len + 2) * RUN_MAX); + if (!p) + return; + } + for (int i = 0; i < len;) { + int n = len - i; + size_t runs_need = (size_t)(n + 1) * RUN_MAX; + p = open_room(h, p, (size_t)n * CELL_BYTES, runs_need, (uint32_t)n * 2); + if (!p || !runs_reserve(&h->runs, runs_need + RUN_MAX)) + return; + size_t used_bytes = (p->off & 0xffff) + p->text_len; + size_t cap = HIST_PAGE_USABLE - used_bytes - (h->runs.len + runs_need + RUN_MAX); + struct enc e; + i += encode_cells(&h->runs, cells + i, n, part_text(h, p) + p->text_len, cap, g, &e); + part_add(h, p, &e); + /* A cluster did not fit: continue the line in a new part. */ + if (i < len && !(p = open_room(h, p, HIST_PAGE_USABLE, 0, 0))) + return; + } + if (!wrapped) + term_hist_close_open_line(h); + else if (h->wrap_cols && (h->walk_on || (p->flags & (HLINE_HAS_WIDE | HLINE_HAS_CLUSTER)))) + walk_advance(h); +} + +void term_hist_push_ascii(struct hist *h, const char *bytes, int n, const struct pen_attr *pen, + bool wrapped) +{ + if (!h->line_limit) + return; + struct pen_attr def = pen_attr_default(); + if (!pen) + pen = &def; + struct hline *p = open_part(h); + if (!p) { + if (n <= 0 && !wrapped) { + term_hist_push_blank(h, 1); + return; + } + /* A default-pen line: one copy and one descriptor. */ + if (!wrapped && h->lines >= h->line_limit) + evict_line(h); + uint32_t pg = h->cur_page; + if (!wrapped && pen_attr_is_default(pen) && pg < h->npages && + h->page_used[pg] + (uint32_t)n <= HIST_PAGE_USABLE && + h->ring && h->head - h->tail <= h->ring_mask) { + uint32_t off = h->page_used[pg]; + bytes_copy_small(h->pages[pg] + off, bytes, n); + h->page_used[pg] = off + (uint32_t)n; + h->page_parts[pg]++; + struct hline *q = part(h, h->head++); + *q = (struct hline){ + .off = (pg << 16) | off, .text_len = (uint32_t)n, .cells = (uint32_t)n, + .flags = HLINE_NARROW, .wrap_rows = ROWS_UNKNOWN, + }; + h->lines++; + h->all_blank = false; + if (h->wrap_cols) + count_closed(h, q, rows_narrow((uint64_t)n, h->wrap_cols)); + return; + } + if (!wrapped && n <= HIST_PAGE_USABLE / 2 && push_line(h, NULL, n, bytes, pen, NULL)) + return; + p = begin_line(h, (size_t)(n < HIST_PAGE_USABLE / 2 ? n : 0) + RUN_MAX * 2); + if (!p) + return; + } + while (n > 0) { + int chunk = n < HIST_PAGE_USABLE / 2 ? n : HIST_PAGE_USABLE / 2; + p = open_room(h, p, (size_t)chunk, RUN_MAX, (uint32_t)chunk); + if (!p || !runs_reserve(&h->runs, RUN_MAX * 2)) + return; + memcpy(part_text(h, p) + p->text_len, bytes, (size_t)chunk); + hist_runs_add(&h->runs, pen, (uint32_t)chunk); + struct enc e = { .bytes = (uint32_t)chunk, .cells = (uint32_t)chunk, + .flags = (uint8_t)(HLINE_NARROW | (pen->link ? HLINE_HAS_LINK : 0)) }; + part_add(h, p, &e); + bytes += chunk; + n -= chunk; + } + if (!wrapped) + term_hist_close_open_line(h); + else if (h->wrap_cols && h->walk_on) + walk_advance(h); +} + +/* Add the runs of a text row to the open part pen by pen, when its bytes + * cannot be copied because the delta bases differ. */ +static void text_row_readd(struct hist *h, const struct text_row *row) +{ + const struct hist_runs *rr = &row->runs; + struct pen_attr pen = row->start; + uint32_t count; + for (const uint8_t *q = rr->data, *end = rr->data + rr->len; q < end;) { + q = read_run(q, &pen, &count); + hist_runs_add(&h->runs, &pen, count); + } + if (rr->count) + hist_runs_add(&h->runs, &rr->pen, rr->count); +} + +void term_hist_push_text(struct hist *h, const struct text_row *row, bool wrapped) +{ + if (!h->line_limit) + return; + const struct hist_runs *rr = &row->runs; + struct hline *p = open_part(h); + size_t runs_need = (size_t)rr->len * RUN_MAX / 2 + 2 * RUN_MAX; + if (!p) { + if (row->cells == 0 && !wrapped) { + term_hist_push_blank(h, 1); + return; + } + if (!wrapped && pen_attr_is_default(&row->start)) { + /* A whole line whose runs start from the default pen: copy the + * text and the runs, then one descriptor. */ + if (h->lines >= h->line_limit) + evict_line(h); + int pg = page_for(h, (size_t)row->text_len + rr->len + RUN_MAX); + if (pg < 0 || !ring_reserve(h)) + return; + uint32_t off = h->page_used[pg]; + uint8_t *dst = h->pages[pg] + off; + bytes_copy_small(dst, (const char *)row->text, row->text_len); + uint8_t *ap = dst + row->text_len; + bytes_copy_small(ap, (const char *)rr->data, (int)rr->len); + uint8_t *end = ap + rr->len; + if (rr->count && !pen_attr_is_default(&rr->pen)) + end = emit_run(end, &rr->base, &rr->pen, rr->count); + struct hline *q = part(h, h->head++); + *q = (struct hline){ + .off = ((uint32_t)pg << 16) | off, + .text_len = row->text_len, + .cells = row->cells, + .wide = row->wide, + .attr_len = (uint16_t)(end - ap), + .flags = row->flags, + .wrap_rows = ROWS_UNKNOWN, + }; + h->page_used[pg] = (uint32_t)(end - h->pages[pg]); + h->page_parts[pg]++; + h->lines++; + h->all_blank = false; + if (h->wrap_cols) + count_closed(h, q, cheap_rows(row->cells, row->wide, h->wrap_cols)); + return; + } + p = begin_line(h, (size_t)row->text_len + runs_need); + if (!p) + return; + } + /* The runs copy when their delta base is the pen of the open part, else + * they are added again pen by pen, which can take more room. */ + runs_flush(&h->runs); + bool copy = pen_attr_equal(&h->runs.base, &row->start); + struct hline *q = open_room(h, p, row->text_len, copy ? rr->len + RUN_MAX : runs_need, + row->cells); + if (!q || !runs_reserve(&h->runs, runs_need + RUN_MAX)) + return; + if (q != p && !(copy = pen_attr_equal(&h->runs.base, &row->start)) && + !(q = open_room(h, q, row->text_len, runs_need, row->cells))) + return; + p = q; + memcpy(part_text(h, p) + p->text_len, row->text, row->text_len); + if (copy) { + memcpy(h->runs.data + h->runs.len, rr->data, rr->len); + h->runs.len += rr->len; + h->runs.base = rr->base; + h->runs.pen = rr->pen; + h->runs.count = rr->count; + } else { + text_row_readd(h, row); + } + part_add(h, p, &(struct enc){ row->text_len, row->cells, row->flags, row->wide }); + if (!wrapped) + term_hist_close_open_line(h); + else if (h->wrap_cols && (h->walk_on || (p->flags & (HLINE_HAS_WIDE | HLINE_HAS_CLUSTER)))) + walk_advance(h); +} + +/* Write the narrow descriptors of ends[i..stop) in blocks, while a block + * fits before the ring wraps. Return the first line left for the caller. */ +#if defined(TERM_KERNEL_AVX512) +TERM_TARGET_AVX512 +static int lf_rows_avx512(struct hist *restrict h, const uint16_t *restrict ends, int i, int stop, + uint32_t pg, uint32_t base, uint32_t cols, uint64_t *head_io, + uint64_t *rows_io, uint32_t *text_count_io) +{ + struct hline *restrict ring = h->ring; + struct hist_block *restrict blocks = h->blocks; + uint32_t mask = h->ring_mask; + uint64_t head = *head_io, rows = *rows_io; + uint32_t text_count = *text_count_io; + const __m512i recip = _mm512_set1_epi32((int)(UINT64_C(0x100000000) / cols + 1)); + for (; i + 16 <= stop && (head & mask) + 16 <= mask + 1; i += 16, head += 16) { + __m512i start = _mm512_cvtepu16_epi32(_mm256_loadu_si256((const __m256i *)(ends + i))); + __m512i end = _mm512_cvtepu16_epi32(_mm256_loadu_si256((const __m256i *)(ends + i + 1))); + __m512i lens = _mm512_sub_epi32(_mm512_sub_epi32(end, start), _mm512_set1_epi32(1)); + __mmask16 empty = _mm512_cmpeq_epi32_mask(lens, _mm512_setzero_si512()); + uint32_t refs = 16 - term_popcount32(empty); + text_count += refs; + __m512i offsets = _mm512_mask_mov_epi32(_mm512_add_epi32(start, _mm512_set1_epi32((int)base)), + empty, _mm512_set1_epi32(-1)); + __m512i rs = _mm512_set1_epi32(1); + uint32_t nr = 16; + if (cols == 1) { + rs = _mm512_max_epu32(lens, rs); + nr = (uint32_t)_mm512_reduce_add_epi32(rs); + } else if (_mm512_cmpgt_epu32_mask(lens, _mm512_set1_epi32((int)cols))) { + /* For a 16-bit numerator, this reciprocal gives exact division. */ + __m512i num = _mm512_max_epi32(_mm512_sub_epi32(lens, _mm512_set1_epi32(1)), + _mm512_setzero_si512()); + __m512i even = _mm512_srli_epi64(_mm512_mul_epu32(num, recip), 32); + __m512i odd = _mm512_mul_epu32(_mm512_srli_epi64(num, 32), recip); + rs = _mm512_add_epi32(_mm512_mask_blend_epi32(0xaaaa, even, odd), rs); + nr = (uint32_t)_mm512_reduce_add_epi32(rs); + } + rows += nr - 16; + blocks[(head & mask) / 16] = (struct hist_block){ + .seq = head + 1, .epoch = h->index_epoch, + .page = pg, .refs = refs, .rows = nr, .cols = h->wrap_cols, + }; + __m512i tags = _mm512_or_si512(_mm512_slli_epi32(_mm512_min_epu32(rs, _mm512_set1_epi32(255)), 24), + _mm512_set1_epi32(HLINE_NARROW << 16)); + __m512i a = _mm512_unpacklo_epi32(offsets, lens); + __m512i b = _mm512_unpackhi_epi32(offsets, lens); + __m512i c = _mm512_unpacklo_epi32(lens, tags); + __m512i d = _mm512_unpackhi_epi32(lens, tags); + __m512i v0 = _mm512_unpacklo_epi64(a, c); + __m512i v1 = _mm512_unpackhi_epi64(a, c); + __m512i v2 = _mm512_unpacklo_epi64(b, d); + __m512i v3 = _mm512_unpackhi_epi64(b, d); + a = _mm512_shuffle_i32x4(v0, v1, 0x44); + b = _mm512_shuffle_i32x4(v2, v3, 0x44); + c = _mm512_shuffle_i32x4(v0, v1, 0xee); + d = _mm512_shuffle_i32x4(v2, v3, 0xee); + struct hline *dst = ring + (head & mask); + _mm512_storeu_si512(dst, _mm512_shuffle_i32x4(a, b, 0x88)); + _mm512_storeu_si512(dst + 4, _mm512_shuffle_i32x4(a, b, 0xdd)); + _mm512_storeu_si512(dst + 8, _mm512_shuffle_i32x4(c, d, 0x88)); + _mm512_storeu_si512(dst + 12, _mm512_shuffle_i32x4(c, d, 0xdd)); + } + *head_io = head; + *rows_io = rows; + *text_count_io = text_count; + return i; +} +#endif + +#if defined(TERM_KERNEL_AVX2) +TERM_TARGET_AVX2 +static int lf_rows_avx2(struct hist *restrict h, const uint16_t *restrict ends, int i, int stop, + uint32_t pg, uint32_t base, uint32_t cols, uint64_t *head_io, + uint64_t *rows_io, uint32_t *text_count_io) +{ + struct hline *restrict ring = h->ring; + struct hist_block *restrict blocks = h->blocks; + uint32_t mask = h->ring_mask; + uint64_t head = *head_io, rows = *rows_io; + uint32_t text_count = *text_count_io; + uint32_t recip = (uint32_t)(UINT64_C(0x100000000) / cols + 1); + uint32_t block_rows = 0, block_refs = 0; + for (; i + 4 <= stop && (head & mask) + 4 <= mask + 1; i += 4, head += 4) { + uint32_t nr = 4, refs; + struct hline *dst = ring + (head & mask); + __m128i start = _mm_cvtepu16_epi32(_mm_loadl_epi64((const __m128i *)(ends + i))); + __m128i end = _mm_cvtepu16_epi32(_mm_loadl_epi64((const __m128i *)(ends + i + 1))); + __m128i lens = _mm_sub_epi32(_mm_sub_epi32(end, start), _mm_set1_epi32(1)); + __m128i empty = _mm_cmpeq_epi32(lens, _mm_setzero_si128()); + refs = 4 - term_popcount32((unsigned)_mm_movemask_ps(_mm_castsi128_ps(empty))); + __m128i offsets = _mm_or_si128(_mm_add_epi32(start, _mm_set1_epi32((int)base)), empty); + __m128i rs = _mm_set1_epi32(1); + if (cols == 1) rs = _mm_max_epu32(lens, rs); + else if (_mm_movemask_ps(_mm_castsi128_ps(_mm_cmpgt_epi32(lens, _mm_set1_epi32((int)cols))))) { + __m128i num = _mm_max_epi32(_mm_sub_epi32(lens, _mm_set1_epi32(1)), _mm_setzero_si128()); + __m128i even = _mm_srli_epi64(_mm_mul_epu32(num, _mm_set1_epi32((int)recip)), 32); + __m128i odd = _mm_mul_epu32(_mm_srli_epi64(num, 32), _mm_set1_epi32((int)recip)); + rs = _mm_add_epi32(_mm_blend_epi16(even, odd, 0xcc), rs); + } + __m128i sum = _mm_hadd_epi32(rs, rs); + nr = (uint32_t)_mm_cvtsi128_si32(_mm_hadd_epi32(sum, sum)); + __m128i tags = _mm_or_si128(_mm_slli_epi32(_mm_min_epu32(rs, _mm_set1_epi32(255)), 24), + _mm_set1_epi32(HLINE_NARROW << 16)); + __m128i a = _mm_unpacklo_epi32(offsets, lens), b = _mm_unpackhi_epi32(offsets, lens); + __m128i c = _mm_unpacklo_epi32(lens, tags), d = _mm_unpackhi_epi32(lens, tags); + _mm_storeu_si128((__m128i *)(dst + 0), _mm_unpacklo_epi64(a, c)); + _mm_storeu_si128((__m128i *)(dst + 1), _mm_unpackhi_epi64(a, c)); + _mm_storeu_si128((__m128i *)(dst + 2), _mm_unpacklo_epi64(b, d)); + _mm_storeu_si128((__m128i *)(dst + 3), _mm_unpackhi_epi64(b, d)); + text_count += refs; + rows += nr - 4; + block_rows += nr; + block_refs += refs; + if ((head & 15) == 12) { + blocks[(head & mask) / 16] = (struct hist_block){ + .seq = head - 12 + 1, .epoch = h->index_epoch, + .page = pg, .refs = block_refs, .rows = block_rows, .cols = h->wrap_cols, + }; + block_rows = block_refs = 0; + } + } + *head_io = head; + *rows_io = rows; + *text_count_io = text_count; + return i; +} +#endif + +static inline int lf_rows(struct hist *restrict h, const uint16_t *restrict ends, int i, int stop, + uint32_t pg, uint32_t base, uint32_t cols, uint64_t *head_io, + uint64_t *rows_io, uint32_t *text_count_io) +{ +#if defined(TERM_KERNEL_AVX512) + if (term_use_avx512()) + return lf_rows_avx512(h, ends, i, stop, pg, base, cols, head_io, rows_io, text_count_io); +#endif +#if defined(TERM_KERNEL_AVX2) + if (term_use_avx2()) + return lf_rows_avx2(h, ends, i, stop, pg, base, cols, head_io, rows_io, text_count_io); +#endif +#if defined(__aarch64__) + struct hline *restrict ring = h->ring; + struct hist_block *restrict blocks = h->blocks; + uint32_t mask = h->ring_mask; + uint64_t head = *head_io, rows = *rows_io; + uint32_t text_count = *text_count_io; + uint32_t recip = (uint32_t)(UINT64_C(0x100000000) / cols + 1); + uint32_t block_rows = 0, block_refs = 0; + for (; i + 4 <= stop && (head & mask) + 4 <= mask + 1; i += 4, head += 4) { + uint32_t nr = 4, refs; + struct hline *dst = ring + (head & mask); + uint32x4_t start = vmovl_u16(vld1_u16(ends + i)); + uint32x4_t end = vmovl_u16(vld1_u16(ends + i + 1)); + uint32x4_t lens = vsubq_u32(vsubq_u32(end, start), vdupq_n_u32(1)); + uint32x4_t empty = vceqq_u32(lens, vdupq_n_u32(0)); + refs = 4 - vaddvq_u32(vshrq_n_u32(empty, 31)); + uint32x4_t offsets = vorrq_u32(vaddq_u32(start, vdupq_n_u32(base)), empty); + uint32x4_t rs = vdupq_n_u32(1); + if (cols == 1) rs = vmaxq_u32(lens, rs); + else if (vmaxvq_u32(lens) > cols) { + uint32x4_t num = vqsubq_u32(lens, vdupq_n_u32(1)); + uint32x2_t lo = vshrn_n_u64(vmull_n_u32(vget_low_u32(num), recip), 32); + uint32x2_t hi = vshrn_n_u64(vmull_n_u32(vget_high_u32(num), recip), 32); + rs = vaddq_u32(vcombine_u32(lo, hi), rs); + } + nr = vaddvq_u32(rs); + uint32x4_t tags = vorrq_u32(vshlq_n_u32(vminq_u32(rs, vdupq_n_u32(255)), 24), + vdupq_n_u32(HLINE_NARROW << 16)); + uint64x2_t a = vreinterpretq_u64_u32(vzip1q_u32(offsets, lens)); + uint64x2_t b = vreinterpretq_u64_u32(vzip2q_u32(offsets, lens)); + uint64x2_t c = vreinterpretq_u64_u32(vzip1q_u32(lens, tags)); + uint64x2_t d = vreinterpretq_u64_u32(vzip2q_u32(lens, tags)); + vst1q_u64((uint64_t *)(void *)(dst + 0), vzip1q_u64(a, c)); + vst1q_u64((uint64_t *)(void *)(dst + 1), vzip2q_u64(a, c)); + vst1q_u64((uint64_t *)(void *)(dst + 2), vzip1q_u64(b, d)); + vst1q_u64((uint64_t *)(void *)(dst + 3), vzip2q_u64(b, d)); + text_count += refs; + rows += nr - 4; + block_rows += nr; + block_refs += refs; + if ((head & 15) == 12) { + blocks[(head & mask) / 16] = (struct hist_block){ + .seq = head - 12 + 1, .epoch = h->index_epoch, + .page = pg, .refs = block_refs, .rows = block_rows, .cols = h->wrap_cols, + }; + block_rows = block_refs = 0; + } + } + *head_io = head; + *rows_io = rows; + *text_count_io = text_count; +#else + (void)h, (void)ends, (void)stop, (void)pg, (void)base, (void)cols; + (void)head_io, (void)rows_io, (void)text_count_io; +#endif + return i; +} + +TERM_NOINLINE +void term_hist_push_ascii_lf(struct hist *restrict h, const char *restrict bytes, + const uint16_t *restrict ends, int count) +{ + if (!h->line_limit) return; + int i = 0; + if (count && hist_is_open(h)) { + term_hist_push_ascii(h, bytes + ends[0], ends[1] - ends[0] - 1, NULL, false); + i++; + } + uint32_t cols = h->wrap_cols ? h->wrap_cols : 1; + while (i < count) { + int stop = count - i > (int)h->line_limit ? i + (int)h->line_limit : count; + if (page_for(h, ends[i + 1] - ends[i]) < 0) return; + uint32_t pg = h->cur_page; + uint32_t room = HIST_PAGE_USABLE - h->page_used[pg]; + if ((uint32_t)(ends[stop] - ends[i]) > room) { + int lo = i + 1, hi = stop; + while (lo < hi) { + int mid = lo + (hi - lo + 1) / 2; + if ((uint32_t)(ends[mid] - ends[i]) <= room) lo = mid; + else hi = mid - 1; + } + stop = lo; + } + uint32_t n = (uint32_t)(stop - i); + evict_for(h, n); + if (!ring_fit(h, n)) return; + bool count_rows = h->wrap_cols && h->rows_id == hist_end_id(h); + uint32_t off = h->page_used[pg]; + uint32_t base = (pg << 16) + off - ends[i]; + uint32_t span = ends[stop] - ends[i]; + memcpy(h->pages[pg] + off, bytes + ends[i], span); + uint64_t head = h->head, rows = n; + struct hline *restrict ring = h->ring; + uint32_t mask = h->ring_mask; + uint32_t text_count = 0; + while (i < stop && (head & 15)) { + uint32_t len = ends[i + 1] - ends[i] - 1; + struct hline *dst = ring + (head & mask); + if (!len) *dst = HLINE_BLANK; + else { + uint32_t r = rows_narrow(len, (int)cols); + part_put_narrow_rows(dst, base + ends[i], len, r); + rows += r - 1; + text_count++; + } + i++; + head++; + } + i = lf_rows(h, ends, i, stop, pg, base, cols, &head, &rows, &text_count); + for (; i < stop; i++, head++) { + uint32_t len = ends[i + 1] - ends[i] - 1; + struct hline *dst = ring + (head & mask); + if (!len) *dst = HLINE_BLANK; + else { + uint32_t r = rows_narrow(len, (int)cols); + part_put_narrow_rows(dst, base + ends[i], len, r); + rows += r - 1; + text_count++; + } + } + h->head = head; + h->page_used[pg] = off + (text_count ? span : 0); + h->page_parts[pg] += text_count; + h->lines += n; + if (text_count) h->all_blank = false; + if (count_rows) { + h->counted_rows += rows; + h->rows_id += n; + } + } +} + +static_assert(sizeof(struct hist_ascii_line) == 16, "a line is two 64-bit lanes"); +static_assert(offsetof(struct hist_ascii_line, len) == 8, "len is in the second 64-bit lane"); + +/* Write the narrow one-row descriptors of lines[i..stream_end) in blocks, + * while a block fits before the ring wraps. first is the first byte of + * the page copy at base. Return the first line left for the caller. */ +#if defined(TERM_KERNEL_AVX512) +TERM_TARGET_AVX512 +static int stream_lines_avx512(struct hist *restrict h, const struct hist_ascii_line *lines, + int i, int stream_end, const char *first, uint32_t base, + uint32_t pg, uint64_t *head_io, uint32_t *text_count_io) +{ + uint64_t head = *head_io; + uint32_t text_count = *text_count_io; + const __m512i even = _mm512_setr_epi64(0, 2, 4, 6, 8, 10, 12, 14); + const __m512i odd = _mm512_setr_epi64(1, 3, 5, 7, 9, 11, 13, 15); + for (; i + 16 <= stream_end && + (head & h->ring_mask) + 16 <= h->ring_mask + 1; i += 16, head += 16) { + __m512i q0 = _mm512_loadu_si512(lines + i); + __m512i q1 = _mm512_loadu_si512(lines + i + 4); + __m512i q2 = _mm512_loadu_si512(lines + i + 8); + __m512i q3 = _mm512_loadu_si512(lines + i + 12); + __m512i p0 = _mm512_permutex2var_epi64(q0, even, q1); + __m512i p1 = _mm512_permutex2var_epi64(q2, even, q3); + __m512i origin = _mm512_set1_epi64((uintptr_t)first); + __m256i lo = _mm512_cvtepi64_epi32(_mm512_sub_epi64(p0, origin)); + __m256i hi = _mm512_cvtepi64_epi32(_mm512_sub_epi64(p1, origin)); + __m512i offsets = _mm512_add_epi32( + _mm512_inserti64x4(_mm512_castsi256_si512(lo), hi, 1), + _mm512_set1_epi32((int)base)); + lo = _mm512_cvtepi64_epi32(_mm512_permutex2var_epi64(q0, odd, q1)); + hi = _mm512_cvtepi64_epi32(_mm512_permutex2var_epi64(q2, odd, q3)); + __m512i lens = _mm512_and_si512( + _mm512_inserti64x4(_mm512_castsi256_si512(lo), hi, 1), + _mm512_set1_epi32(UINT16_MAX)); + __mmask16 empty = _mm512_cmpeq_epi32_mask(lens, _mm512_setzero_si512()); + uint32_t refs = 16 - term_popcount32(empty); + text_count += refs; + offsets = _mm512_mask_mov_epi32(offsets, empty, _mm512_set1_epi32(-1)); + __m512i tags = _mm512_set1_epi32((1u << 24) | (HLINE_NARROW << 16)); + __m512i a = _mm512_unpacklo_epi32(offsets, lens); + __m512i b = _mm512_unpackhi_epi32(offsets, lens); + __m512i c = _mm512_unpacklo_epi32(lens, tags); + __m512i d = _mm512_unpackhi_epi32(lens, tags); + __m512i v0 = _mm512_unpacklo_epi64(a, c); + __m512i v1 = _mm512_unpackhi_epi64(a, c); + __m512i v2 = _mm512_unpacklo_epi64(b, d); + __m512i v3 = _mm512_unpackhi_epi64(b, d); + a = _mm512_shuffle_i32x4(v0, v1, 0x44); + b = _mm512_shuffle_i32x4(v2, v3, 0x44); + c = _mm512_shuffle_i32x4(v0, v1, 0xee); + d = _mm512_shuffle_i32x4(v2, v3, 0xee); + struct hline *dst = h->ring + (head & h->ring_mask); + _mm512_storeu_si512(dst, _mm512_shuffle_i32x4(a, b, 0x88)); + _mm512_storeu_si512(dst + 4, _mm512_shuffle_i32x4(a, b, 0xdd)); + _mm512_storeu_si512(dst + 8, _mm512_shuffle_i32x4(c, d, 0x88)); + _mm512_storeu_si512(dst + 12, _mm512_shuffle_i32x4(c, d, 0xdd)); + if (!(head & 15)) { + h->blocks[(head & h->ring_mask) / 16] = (struct hist_block){ + .seq = head + 1, .epoch = h->index_epoch, + .page = pg, .refs = refs, .rows = 16, .cols = h->wrap_cols, + }; + } + } + *head_io = head; + *text_count_io = text_count; + return i; +} +#endif + +#if defined(TERM_KERNEL_AVX2) +TERM_TARGET_AVX2 +static int stream_lines_avx2(struct hist *restrict h, const struct hist_ascii_line *lines, + int i, int stream_end, const char *first, uint32_t base, + uint32_t pg, uint64_t *head_io, uint32_t *text_count_io) +{ + (void)pg; + uint64_t head = *head_io; + uint32_t text_count = *text_count_io; + for (; i + 4 <= stream_end && + (head & h->ring_mask) + 4 <= h->ring_mask + 1; i += 4, head += 4) { + struct hline *dst = h->ring + (head & h->ring_mask); + uint32_t refs; + __m128i index = _mm_setr_epi32(0, 16, 32, 48); + __m128i ptrs = _mm_i32gather_epi32((const int *)(lines + i), index, 1); + __m128i lens = _mm_and_si128( + _mm_i32gather_epi32((const int *)((const char *)(lines + i) + 8), index, 1), + _mm_set1_epi32(UINT16_MAX)); + __m128i empty = _mm_cmpeq_epi32(lens, _mm_setzero_si128()); + refs = 4 - term_popcount32((unsigned)_mm_movemask_ps(_mm_castsi128_ps(empty))); + __m128i offsets = _mm_add_epi32(ptrs, + _mm_set1_epi32((int)(base - (uint32_t)(uintptr_t)first))); + offsets = _mm_or_si128(offsets, empty); + __m128i tags = _mm_set1_epi32((1u << 24) | (HLINE_NARROW << 16)); + __m128i a = _mm_unpacklo_epi32(offsets, lens); + __m128i b = _mm_unpackhi_epi32(offsets, lens); + __m128i c = _mm_unpacklo_epi32(lens, tags); + __m128i d = _mm_unpackhi_epi32(lens, tags); + _mm256_storeu_si256((__m256i *)dst, _mm256_inserti128_si256( + _mm256_castsi128_si256(_mm_unpacklo_epi64(a, c)), _mm_unpackhi_epi64(a, c), 1)); + _mm256_storeu_si256((__m256i *)(dst + 2), _mm256_inserti128_si256( + _mm256_castsi128_si256(_mm_unpacklo_epi64(b, d)), _mm_unpackhi_epi64(b, d), 1)); + text_count += refs; + } + *head_io = head; + *text_count_io = text_count; + return i; +} +#endif + +static inline int stream_lines(struct hist *restrict h, const struct hist_ascii_line *lines, + int i, int stream_end, const char *first, uint32_t base, + uint32_t pg, uint64_t *head_io, uint32_t *text_count_io) +{ +#if defined(TERM_KERNEL_AVX512) + if (term_use_avx512()) + return stream_lines_avx512(h, lines, i, stream_end, first, base, pg, head_io, text_count_io); +#endif +#if defined(TERM_KERNEL_AVX2) + if (term_use_avx2()) + return stream_lines_avx2(h, lines, i, stream_end, first, base, pg, head_io, text_count_io); +#endif +#if defined(__aarch64__) + (void)pg; + uint64_t head = *head_io; + uint32_t text_count = *text_count_io; + for (; i + 4 <= stream_end && + (head & h->ring_mask) + 4 <= h->ring_mask + 1; i += 4, head += 4) { + struct hline *dst = h->ring + (head & h->ring_mask); + uint32_t refs; + uint32x4x4_t q = vld4q_u32((const uint32_t *)(lines + i)); + uint32x4_t lens = vandq_u32(q.val[2], vdupq_n_u32(UINT16_MAX)); + uint32x4_t empty = vceqq_u32(lens, vdupq_n_u32(0)); + refs = 4 - vaddvq_u32(vshrq_n_u32(empty, 31)); + uint32x4_t offsets = vorrq_u32(empty, vaddq_u32(q.val[0], + vdupq_n_u32(base - (uint32_t)(uintptr_t)first))); + uint32x4x4_t parts = { .val = { + offsets, lens, lens, vdupq_n_u32((1u << 24) | (HLINE_NARROW << 16)), + } }; + vst4q_u32((uint32_t *)dst, parts); + text_count += refs; + } + *head_io = head; + *text_count_io = text_count; +#else + (void)h, (void)lines, (void)stream_end, (void)first, (void)base, (void)pg; + (void)head_io, (void)text_count_io; +#endif + return i; +} + +void term_hist_push_ascii_lines(struct hist *h, const struct hist_ascii_line *lines, int count, + const struct pen_attr *pen) +{ + if (!h->line_limit) + return; + bool def = !pen || pen_attr_is_default(pen); + uint16_t cols = h->wrap_cols ? h->wrap_cols : 1; + for (int i = 0; i < count;) { + const struct hist_ascii_line *l = &lines[i]; + if (l->wrapped) { + int end = i + 1; + int bytes = l->len; + while (end < count && lines[end].wrapped && + lines[end].ptr == l->ptr + bytes && + lines[end].len <= INT32_MAX - bytes) { + bytes += lines[end++].len; + } + term_hist_push_ascii(h, l->ptr, bytes, pen, true); + i = end; + continue; + } + if (!def || hist_is_open(h)) { + if (l->len == 0 && !l->wrapped) + term_hist_push_blank(h, 1); + else + term_hist_push_ascii(h, l->ptr, l->len, pen, l->wrapped); + i++; + continue; + } + /* A batch of unwrapped default-pen lines that fit the current page: + * one copy and one descriptor each. A blank line joins the batch + * instead of ending it. */ + if (page_for(h, l->len) < 0) + return; + uint32_t pg = h->cur_page; + uint32_t room = HIST_PAGE_USABLE - h->page_used[pg]; + int end = i; + uint32_t bytes = 0; + /* Keep separators in the page to copy a contiguous input batch once. */ + int stream_end = i; + uint32_t extra_rows = 0; + while (stream_end < count && stream_end - i < (int)h->line_limit) { + const struct hist_ascii_line *next = &lines[stream_end]; + if (next->wrapped || next->len > HIST_PAGE_USABLE / 2 || + (stream_end > i && + (next->ptr != lines[stream_end - 1].ptr + lines[stream_end - 1].len + 1 || + next->ptr[-1] != '\n'))) + break; + size_t span = (size_t)(next->ptr - l->ptr) + next->len; + if (span > room) + break; + if (TERM_EXPECT(next->len > (uint32_t)cols, 0)) + extra_rows += rows_narrow(next->len, cols) - 1; + stream_end++; + } + if (stream_end - i >= 4) { + uint32_t n = (uint32_t)(stream_end - i); + uint64_t stream_rows = (uint64_t)n + extra_rows; + const struct hist_ascii_line *last = &lines[stream_end - 1]; + uint32_t span = (uint32_t)(last->ptr - l->ptr) + last->len; + evict_for(h, n); + if (!ring_fit(h, n)) + return; + uint32_t off = h->page_used[pg]; + memcpy(h->pages[pg] + off, l->ptr, span); + uint32_t base = (pg << 16) | off; + uint32_t text_count = 0; + uint64_t head = h->head; + if (!extra_rows) + i = stream_lines(h, lines, i, stream_end, l->ptr, base, pg, &head, &text_count); + for (; i < stream_end; i++) { + uint32_t len = lines[i].len; + struct hline *dst = &h->ring[head++ & h->ring_mask]; + if (len) { + uint32_t line_off = base + (uint32_t)(lines[i].ptr - l->ptr); + if (TERM_EXPECT(len <= cols, 1)) { + part_put_narrow(dst, line_off, len); + } else { + part_put_narrow_rows(dst, line_off, len, rows_narrow(len, cols)); + } + text_count++; + } else { + *dst = HLINE_BLANK; + } + } + if (h->wrap_cols && h->rows_id == hist_end_id(h)) { + h->counted_rows += stream_rows; + h->rows_id += n; + } + h->head = head; + h->page_used[pg] = off + span; + h->page_parts[pg] += text_count; + h->lines += n; + if (text_count) + h->all_blank = false; + continue; + } + /* line_limit does not move inside this scan. */ + int batch_max = (int)h->line_limit; + while (end < count && !lines[end].wrapped && + bytes + lines[end].len <= room && end - i < batch_max) { + bytes += lines[end].len; + end++; + } + uint32_t n = (uint32_t)(end - i); + evict_for(h, n); + if (!ring_fit(h, n)) + return; + bool count_rows = h->wrap_cols && h->rows_id == hist_end_id(h); + uint64_t rows_add = 0; + /* Lines that took a page slot. A blank line takes none. */ + uint32_t n_text = 0; + bool any_text = false; + uint32_t off = h->page_used[pg]; + uint8_t *page = h->pages[pg]; + /* The ring and head cannot move inside this loop: eviction and the + * ring growth both ran above. Keep them in registers. */ + struct hline *ring = h->ring; + uint32_t rmask = h->ring_mask; + uint64_t head = h->head; + uint32_t page_off = pg << 16; + for (; i < end; i++) { + uint32_t len = lines[i].len; + /* A blank line: no text, no page slot, one row. The scan does + * not stop on it, so the batch keeps its room and its setup. */ + if (TERM_EXPECT(len == 0, 0)) { + ring[head++ & rmask] = HLINE_BLANK; + rows_add++; + continue; + } + bytes_copy_small(page + off, lines[i].ptr, (int)len); + n_text++; + any_text = true; + if (TERM_EXPECT(len <= cols, 1)) { + part_put_narrow(&ring[head++ & rmask], page_off | off, len); + rows_add++; + } else { + uint32_t rows = rows_narrow(len, cols); + part_put_narrow_rows(&ring[head++ & rmask], page_off | off, len, rows); + rows_add += rows; + } + off += len; + } + h->head = head; + h->page_used[pg] = off; + h->page_parts[pg] += n_text; + h->lines += n; + if (any_text) + h->all_blank = false; + if (count_rows) { + h->counted_rows += rows_add; + h->rows_id += n; + } + } +} + +void term_hist_push_blank(struct hist *h, size_t n) +{ + if (!h->line_limit || n == 0) + return; + if (hist_is_open(h)) { + term_hist_close_open_line(h); + n--; + } + if (n == 0) + return; + if (h->all_blank && h->lines == h->line_limit) { + /* Every line is blank and has one row: only the ids move. */ + bool all = h->rows_id == hist_end_id(h); + h->base_id += n; + if (all || h->rows_id < h->base_id) + h->rows_id = all ? hist_end_id(h) : h->base_id; + h->counted_rows = h->rows_id - h->base_id; + return; + } + if (n >= h->line_limit) { + uint64_t end = hist_end_id(h) + n; + term_hist_clear(h); + h->base_id = h->rows_id = end - h->line_limit; + n = h->line_limit; + h->all_blank = true; + } + evict_for(h, n); + /* Ring room for the whole run at once. Growing it later would + * invalidate the pointers the loop below keeps in registers. */ + if (!ring_fit(h, n)) + return; + bool count = h->wrap_cols && h->rows_id == hist_end_id(h); + /* The ring and head cannot move here: eviction ran above and the + * reserve cannot grow the ring again. Keep both in registers. */ + struct hline *ring = h->ring; + uint32_t rmask = h->ring_mask; + uint64_t head = h->head; + for (size_t i = 0; i < n; i++) + ring[head++ & rmask] = HLINE_BLANK; + h->head = head; + h->lines += n; + if (count) { + h->rows_id += n; + h->counted_rows += n; + } +} + +/* ---- Reading ---- */ + +/* Walks the attribute runs of one part, cell by cell. */ +struct runs_iter { + const uint8_t *p, *end; + bool pending; /* the open part: the pending run follows */ + struct pen_attr pending_pen; + uint32_t pending_count; + struct pen_attr pen; + uint32_t left; /* cells left in the run; UINT32_MAX: default tail */ +}; + +/* Runs held in a hist_runs (the open part, or a text row) from pen start. */ +static void runs_iter_runs(struct runs_iter *it, const struct hist_runs *r, const struct pen_attr *start) +{ + *it = (struct runs_iter){ + .p = r->data, .end = r->data ? r->data + r->len : NULL, + .pending = r->count > 0, .pending_pen = r->pen, .pending_count = r->count, + .pen = *start, + }; +} + +static void runs_iter_init(struct runs_iter *it, const struct hist *h, const struct hline *p) +{ + *it = (struct runs_iter){ .pen = pen_attr_default() }; + if (p->flags & HLINE_OPEN) { + if (h->runs.len) { + it->p = h->runs.data; + it->end = h->runs.data + h->runs.len; + } + it->pending = h->runs.count > 0; + it->pending_pen = h->runs.pen; + it->pending_count = h->runs.count; + } else if (p->attr_len) { + it->p = part_text(h, p) + p->text_len; + it->end = it->p + p->attr_len; + } +} + +/* Take up to n cells of the current run. it->pen is their pen. */ +static inline uint32_t runs_take(struct runs_iter *it, uint32_t n) +{ + while (it->left == 0) { + if (it->p < it->end) { + it->p = read_run(it->p, &it->pen, &it->left); + } else if (it->pending) { + it->pending = false; + it->pen = it->pending_pen; + it->left = it->pending_count; + } else { + it->pen = pen_attr_default(); + it->left = UINT32_MAX; + } + } + if (it->left == UINT32_MAX) + return n; + uint32_t k = it->left < n ? it->left : n; + it->left -= k; + return k; +} + +static inline void runs_iter_save(const struct runs_iter *it, struct hist_dec_cache *d) +{ + d->p = it->p; + d->end = it->end; + d->pending = it->pending; + d->pending_pen = it->pending_pen; + d->pending_count = it->pending_count; + d->pen = it->pen; + d->left = it->left; +} + +static inline void runs_iter_restore(struct runs_iter *it, const struct hist_dec_cache *d) +{ + it->p = d->p; + it->end = d->end; + it->pending = d->pending; + it->pending_pen = d->pending_pen; + it->pending_count = d->pending_count; + it->pen = d->pen; + it->left = d->left; +} + +static inline int text_width(const uint8_t *s) +{ + uint8_t b = s[0]; + if (b < PREFIX_WIDE || b == PREFIX_TEXT) + return 1; + if (b != PREFIX_CLUSTER) + return 2; + return (s[1] & 0x80) ? 2 : 1; +} + +static inline const uint8_t *text_skip(const uint8_t *s, const uint8_t *end) +{ + uint8_t b = s[0]; + uint32_t cp; + if (b < 0x80) + return s + 1; + if (b < PREFIX_WIDE) + return s + get_utf8(s, end, &cp); + if (b != PREFIX_CLUSTER) + return s + 1 + get_utf8(s + 1, end, &cp); + uint8_t k = s[1]; + s += 2; + for (int i = 0; i < (k & 0x1f) && s < end; i++) + s += get_utf8(s, end, &cp); + return s; +} + +/* Read one cell: its code point (a handle for a cluster when g is set) and + * its shape flags. */ +static const uint8_t *text_cell(const uint8_t *s, const uint8_t *end, struct gstore *g, + uint32_t *cp, uint16_t *shape) +{ + uint8_t b = s[0]; + *shape = 0; + if (b < 0x80) { + *cp = b; + return s + 1; + } + if (b < PREFIX_WIDE) + return s + get_utf8(s, end, cp); + if (b != PREFIX_CLUSTER) { + *shape = b == PREFIX_WIDE ? CELL_WIDE : b == PREFIX_EMOJI ? (CELL_WIDE | CELL_EMOJI) : CELL_TEXT; + return s + 1 + get_utf8(s + 1, end, cp); + } + uint8_t k = s[1]; + s += 2; + uint32_t cps[GRAPHEME_MAX_LEN]; + int n = 0; + for (int i = 0; i < (k & 0x1f) && s < end; i++) { + uint32_t c; + s += get_utf8(s, end, &c); + if (n < GRAPHEME_MAX_LEN) + cps[n++] = c; + } + *cp = n == 0 ? ' ' : (g && n > 1) ? term_gstore_intern(g, cps, n) : cps[0]; + *shape = (uint16_t)(((k & 0x80) ? CELL_WIDE : 0) | ((k & 0x40) ? CELL_EMOJI : 0) | + ((k & 0x20) ? CELL_TEXT : 0)); + return s; +} + +uint64_t term_hist_line_cells(const struct hist *h, uint64_t id) +{ + if (!line_valid(h, id)) + return 0; + if (id == h->last_id && h->last_cells) + return h->last_cells; + uint64_t first = line_first(h, id), end = line_end(h, first), cells = 0; + for (uint64_t s = first; s < end; s++) + cells += part(h, s)->cells; + return cells; +} + +uint64_t term_hist_line_wide(const struct hist *h, uint64_t id) +{ + if (!line_valid(h, id)) + return 0; + if (id == h->last_id && h->last_cells) + return h->last_wide; + uint64_t first = line_first(h, id), end = line_end(h, first), wide = 0; + for (uint64_t seq = first; seq < end; seq++) + wide += part(h, seq)->wide; + return wide; +} + +uint8_t term_hist_line_flags(const struct hist *h, uint64_t id) +{ + if (!line_valid(h, id)) + return 0; + uint64_t first = line_first(h, id), end = line_end(h, first); + uint8_t flags = part(h, first)->flags; + for (uint64_t s = first + 1; s < end; s++) { + uint8_t f = part(h, s)->flags; + flags = (uint8_t)((flags | f) & ~HLINE_NARROW) | (flags & f & HLINE_NARROW); + } + return flags; +} + +static int line_rows_walk(const struct hist *h, uint64_t first, uint64_t end, int cols) +{ + int rows = 1, col = 0; + for (uint64_t s = first; s < end; s++) { + const struct hline *p = part(h, s); + if (!p->text_len) + continue; + const uint8_t *t = part_text(h, p), *end = t + p->text_len; + while (t < end) { + int w = text_width(t) == 2 && cols > 1 ? 2 : 1; + t = text_skip(t, end); + if (col == cols || (w == 2 && col + 2 > cols)) { + rows++; + col = 0; + } + col += w; + } + } + return rows; +} + +int term_hist_line_rows(const struct hist *h, uint64_t id, int cols) +{ + if (cols <= 0 || !line_valid(h, id)) + return 0; + uint64_t first = line_first(h, id); + const struct hline *f = part(h, first); + if (cols == h->wrap_cols) { + if (id < h->rows_id && + ((f->flags & HLINE_ROWS_EXACT) || !(term_hist_line_flags(h, id) & HLINE_HAS_WIDE))) { + if (f->wrap_rows != ROWS_UNKNOWN) + return f->wrap_rows; + if ((f->flags & HLINE_ROWS_EXACT) && h->exact_rows) + return (int)h->exact_rows[first & h->ring_mask]; + } + if (h->walk_on && id + 1 == hist_end_id(h) && h->walk_seq + 1 == h->head && + h->walk_off == part(h, h->walk_seq)->text_len) + return (int)h->walk_rows; + } + uint64_t cells = term_hist_line_cells(h, id); + if (cells <= (uint64_t)cols || + !(term_hist_line_flags(h, id) & (HLINE_HAS_WIDE | HLINE_HAS_CLUSTER))) + return (int)rows_narrow(cells, cols); + return line_rows_walk(h, first, line_end(h, first), cols); +} + +/* Walk the open line from where the last walk stopped. */ +static void walk_advance(struct hist *h) +{ + int cols = h->wrap_cols; + if (!h->walk_on) { + h->walk_seq = line_first(h, hist_end_id(h) - 1); + h->walk_off = 0; + h->walk_col = 0; + h->walk_rows = 1; + h->walk_on = true; + } + for (;;) { + const struct hline *p = part(h, h->walk_seq); + if (p->text_len) { + const uint8_t *t0 = part_text(h, p), *t = t0 + h->walk_off, *end = t0 + p->text_len; + uint32_t col = h->walk_col, rows = h->walk_rows; + while (t < end) { + uint32_t w = text_width(t) == 2 && cols > 1 ? 2 : 1; + t = text_skip(t, end); + if (col == (uint32_t)cols || (w == 2 && col + 2 > (uint32_t)cols)) { + rows++; + col = 0; + } + col += w; + } + h->walk_col = col; + h->walk_rows = rows; + h->walk_off = (uint32_t)(t - t0); + } + if (h->walk_seq + 1 >= h->head) + break; + h->walk_seq++; + h->walk_off = 0; + } +} + +static inline void put_cell(struct cell *out, uint32_t cp, uint16_t shape, const struct pen_attr *pen) +{ + *out = (struct cell){ cp, pen->fg, pen->bg, (uint16_t)(shape | pen->flags), pen->link }; +} + +/* A line is narrow when every part stores one ASCII byte per cell. Then a + * cell offset is also a byte offset, in every part. */ +static bool line_narrow(const struct hist *h, uint64_t first, uint64_t end) +{ + for (uint64_t s = first; s < end; s++) + if (!(part(h, s)->flags & HLINE_NARROW)) + return false; + return true; +} + +int term_hist_decode_segments(const struct hist *h, uint64_t id, int start_seg, int count, int cols, + struct cell *out, int *lens, struct gstore *g) +{ + if (count <= 0 || start_seg < 0 || cols <= 0 || !line_valid(h, id)) + return 0; + uint64_t first = line_first(h, id), end = line_end(h, first); + + /* A narrow line has one byte per cell, so a segment starts at a known + * cell offset. Walk the parts to reach it, not the text before it. */ + if (line_narrow(h, first, end)) { + uint64_t skip = (uint64_t)start_seg * (uint64_t)cols; + int seg_idx = 0; + int n = 0; + for (uint64_t s = first; s < end; s++) { + const struct hline *p = part(h, s); + if (skip >= p->cells) { + skip -= p->cells; + continue; + } + uint64_t avail = p->cells - skip; + struct runs_iter it; + runs_iter_init(&it, h, p); + for (uint64_t k = skip; k > 0;) + k -= runs_take(&it, k > UINT32_MAX ? UINT32_MAX : (uint32_t)k); + const char *src = (const char *)part_text(h, p) + skip; + while (avail > 0 && seg_idx < count) { + int need = cols - n; + int take = avail > (uint64_t)need ? need : (int)avail; + struct cell *row_out = out + (size_t)seg_idx * (size_t)cols; + for (int x = 0; x < take;) { + int k = (int)runs_take(&it, (uint32_t)(take - x)); + put_ascii_chunk(row_out + n + x, src + x, k, it.pen.fg, it.pen.bg, + (uint32_t)it.pen.flags | ((uint32_t)it.pen.link << 16)); + x += k; + } + n += take; + src += take; + avail -= (uint64_t)take; + if (n == cols) { + if (lens) + lens[seg_idx] = cols; + seg_idx++; + n = 0; + } + } + skip = 0; + if (seg_idx == count) + return count; + } + if (seg_idx < count) { + if (lens) + lens[seg_idx] = n; + seg_idx++; + } + return seg_idx; + } + + if (g) + term_gstore_pin(g); + int row = 0, col = 0; + int end_seg = start_seg + count; + uint64_t s = first; + struct runs_iter it; + struct hist_dec_cache *dec = &((struct hist *)(uintptr_t)h)->dec; + bool resumed = false; + + if (dec->valid && dec->epoch == h->index_epoch && dec->id == id && + dec->cols == cols && dec->seg == (uint32_t)start_seg && + dec->seq >= first && dec->seq < end && + dec->off <= part(h, dec->seq)->text_len) { + s = dec->seq; + row = start_seg; + col = (int)dec->col; + runs_iter_restore(&it, dec); + resumed = true; + } + + for (; s < end; s++) { + const struct hline *p = part(h, s); + if (!p->text_len) + continue; + const uint8_t *t = part_text(h, p), *tend = t + p->text_len; + if (resumed && s == dec->seq) { + t += dec->off; + resumed = false; + } else { + uint32_t skipped_cells = 0; + if (row < start_seg) { + while (t < tend && row < start_seg) { + int w = text_width(t) == 2 && cols > 1 ? 2 : 1; + if (col == cols || (w == 2 && col + 2 > cols)) { + row++; + col = 0; + if (row == start_seg) + break; + } + skipped_cells++; + col += w; + t = text_skip(t, tend); + } + if (row < start_seg) + continue; + } + + runs_iter_init(&it, h, p); + for (uint32_t k = skipped_cells; k > 0;) + k -= runs_take(&it, k > UINT32_MAX ? UINT32_MAX : k); + } + + while (t < tend && row < end_seg) { + int w = text_width(t) == 2 && cols > 1 ? 2 : 1; + if (col == cols || (w == 2 && col + 2 > cols)) { + int seg_idx = row - start_seg; + struct cell *row_out = out + (size_t)seg_idx * (size_t)cols; + if (col < cols) + row_out[col] = (struct cell){ ' ', TERM_COLOR_FG, TERM_COLOR_BG, CELL_WRAP_PAD, 0 }; + if (lens) + lens[seg_idx] = col < cols ? cols : col; + row++; + col = 0; + if (row == end_seg) { + if (g) + term_gstore_unpin(g); + dec->epoch = h->index_epoch; + dec->id = id; + dec->cols = (uint16_t)cols; + dec->seq = s; + dec->seg = (uint32_t)end_seg; + dec->off = (uint32_t)(t - part_text(h, p)); + dec->col = 0; + runs_iter_save(&it, dec); + dec->valid = true; + return count; + } + } + int seg_idx = row - start_seg; + struct cell *row_out = out + (size_t)seg_idx * (size_t)cols; + runs_take(&it, 1); + uint32_t cp; + uint16_t shape; + t = text_cell(t, tend, g, &cp, &shape); + put_cell(&row_out[col], cp, shape, &it.pen); + if (w == 2) + put_cell(&row_out[col + 1], 0, (uint16_t)((shape & ~CELL_WIDE) | CELL_TRAIL), &it.pen); + col += w; + } + if (row == end_seg) { + if (g) + term_gstore_unpin(g); + dec->epoch = h->index_epoch; + dec->id = id; + dec->cols = (uint16_t)cols; + dec->seq = s; + dec->seg = (uint32_t)end_seg; + dec->off = (uint32_t)(t - part_text(h, p)); + dec->col = (uint32_t)col; + runs_iter_save(&it, dec); + dec->valid = true; + return count; + } + } + int decoded = row - start_seg; + if (row >= start_seg && row < end_seg) { + if (lens) + lens[decoded] = col; + decoded++; + } + dec->valid = false; + if (g) + term_gstore_unpin(g); + return decoded; +} + +int term_hist_decode_segment(const struct hist *h, uint64_t id, int seg, int cols, + struct cell *out, struct gstore *g) +{ + int len = 0; + int got = term_hist_decode_segments(h, id, seg, 1, cols, out, &len, g); + return got > 0 ? len : 0; +} + +/* Write the cells of parts [first, end) in one row, without wrapping. */ +static int decode_line(const struct hist *h, uint64_t first, uint64_t end, + struct cell *out, int cap, struct gstore *g) +{ + int n = 0; + if (g) + term_gstore_pin(g); + for (uint64_t s = first; s < end; s++) { + const struct hline *p = part(h, s); + if (!p->text_len) + continue; + struct runs_iter it; + runs_iter_init(&it, h, p); + const uint8_t *t = part_text(h, p), *tend = t + p->text_len; + while (t < tend) { + uint32_t cp; + uint16_t shape; + runs_take(&it, 1); + t = text_cell(t, tend, g, &cp, &shape); + int w = (shape & CELL_WIDE) ? 2 : 1; + if (n + w <= cap) { + put_cell(&out[n], cp, shape, &it.pen); + if (w == 2) + put_cell(&out[n + 1], 0, (uint16_t)((shape & ~CELL_WIDE) | CELL_TRAIL), &it.pen); + } + n += w; + } + } + if (g) + term_gstore_unpin(g); + return n; +} + +int term_hist_text_decode(const struct text_row *row, struct cell *out, int cap, struct gstore *g) +{ + struct runs_iter it; + runs_iter_runs(&it, &row->runs, &row->start); + if (row->flags & HLINE_NARROW) { + /* One byte per cell: one block store per run. */ + int take = row->text_len < cap ? row->text_len : cap; + for (int x = 0; x < take;) { + int k = (int)runs_take(&it, (uint32_t)(take - x)); + put_ascii_chunk(out + x, (const char *)row->text + x, k, it.pen.fg, it.pen.bg, + (uint32_t)it.pen.flags | ((uint32_t)it.pen.link << 16)); + x += k; + } + return take; + } + const uint8_t *t = row->text, *tend = row->text + row->text_len; + int n = 0; + while (t < tend) { + uint32_t cp; + uint16_t shape; + runs_take(&it, 1); + t = text_cell(t, tend, g, &cp, &shape); + int w = (shape & CELL_WIDE) ? 2 : 1; + if (n + w <= cap) { + put_cell(&out[n], cp, shape, &it.pen); + if (w == 2) + put_cell(&out[n + 1], 0, (uint16_t)((shape & ~CELL_WIDE) | CELL_TRAIL), &it.pen); + } else if (n < cap) { + put_cell(&out[n], cp, shape, &it.pen); + } + n += w; + } + return n < cap ? n : cap; +} + +void term_hist_text_mark_links(const struct text_row *row, void (*mark)(void *ud, uint16_t link), void *ud) +{ + if (!(row->flags & HLINE_HAS_LINK) && !row->start.link) + return; + struct runs_iter it; + runs_iter_runs(&it, &row->runs, &row->start); + struct pen_attr pen = row->start; + uint32_t count; + if (pen.link) + mark(ud, pen.link); + while (it.p < it.end) { + it.p = read_run(it.p, &pen, &count); + if (pen.link) + mark(ud, pen.link); + } + if (it.pending && it.pending_pen.link) + mark(ud, it.pending_pen.link); +} + +void term_hist_mark_links(const struct hist *h, void (*mark)(void *ud, uint16_t link), void *ud) +{ + for (uint64_t s = h->tail; s < h->head; s++) { + const struct hline *p = part(h, s); + if (!(p->flags & HLINE_HAS_LINK)) + continue; + struct runs_iter it; + runs_iter_init(&it, h, p); + struct pen_attr pen = pen_attr_default(); + uint32_t count; + while (it.p < it.end) { + it.p = read_run(it.p, &pen, &count); + if (pen.link) + mark(ud, pen.link); + } + if (it.pending && it.pending_pen.link) + mark(ud, it.pending_pen.link); + } +} + +/* ---- Removing from the end ---- */ + +static void drop_head_part(struct hist *h) +{ + struct hline *p = part(h, --h->head); + if (p->off != NO_TEXT) { + uint32_t pg = p->off >> 16; + uint32_t start = p->off & 0xffff; + uint32_t stop = start + p->text_len + ((p->flags & HLINE_OPEN) ? 0 : p->attr_len); + if (pg == h->cur_page && h->page_used[pg] == stop) + h->page_used[pg] = start; + page_drop_ref(h, pg); + } + cont_trim(h); +} + +int term_hist_peek_last_line(const struct hist *h, struct cell *out, int cap, struct gstore *g) +{ + if (h->lines == 0) + return 0; + uint64_t id = hist_end_id(h) - 1; + return decode_line(h, line_first(h, id), h->head, out, cap, g); +} + +int term_hist_pop_line(struct hist *h, struct cell *out, int cap, struct gstore *g) +{ + if (h->lines == 0) + return 0; + uint64_t id = hist_end_id(h) - 1; + uint64_t first = line_first(h, id); + int n = out ? decode_line(h, first, h->head, out, cap, g) : (int)term_hist_line_cells(h, id); + if (id < h->rows_id) { + rows_uncount(h, id); + h->rows_id = id; + } + while (h->head > first) + drop_head_part(h); + h->lines--; + h->index_epoch++; + if (h->lines) { + uint64_t last = hist_end_id(h) - 1; + h->last_cells = term_hist_line_cells(h, last); + h->last_wide = term_hist_line_wide(h, last); + h->last_id = last; + } + runs_reset(&h->runs); + h->walk_on = false; + if (h->lines == 0 && h->head == h->tail) { + h->all_blank = true; + h->base_cell = 0; + h->last_cells = h->last_wide = 0; + } + return n; +} + +int term_hist_pop_tail_cells(struct hist *h, int n, struct cell *out, struct gstore *g) +{ + if (h->lines == 0 || n <= 0) + return 0; + uint64_t total = term_hist_line_cells(h, hist_end_id(h) - 1); + if (total == 0) { + term_hist_pop_line(h, NULL, 0, g); + return 0; + } + struct cell *all = malloc((size_t)total * sizeof(*all)); + if (!all) + return 0; + if (g) + term_gstore_pin(g); + int got = term_hist_pop_line(h, all, (int)total, g); + int take = n < got ? n : got; + int keep = got - take; + if (keep > 0 && (all[keep].flags & CELL_TRAIL)) { + keep++; + take--; + } + if (out && take > 0) + memcpy(out, all + keep, (size_t)take * sizeof(*out)); + if (keep > 0) { + uint16_t cols = h->wrap_cols ? h->wrap_cols : 1; + for (int i = 0; i < keep; i += cols) + term_hist_push_row(h, all + i, keep - i < cols ? keep - i : cols, true, g); + } + if (g) + term_gstore_unpin(g); + free(all); + return take; +} + +/* ---- Rows ---- */ + +void term_hist_set_width(struct hist *h, int cols) +{ + if (h->wrap_cols == cols) + return; + h->wrap_cols = (uint16_t)cols; + h->rows_id = h->base_id; + h->counted_rows = 0; + h->walk_on = false; +} + +uint64_t term_hist_line_estimate(const struct hist *h, uint64_t id, int cols) +{ + if (!line_valid(h, id)) + return 0; + if (cols == h->wrap_cols && id < h->rows_id) { + const struct hline *first = part(h, line_first(h, id)); + if (first->flags & HLINE_ROWS_EXACT) + return first->wrap_rows != ROWS_UNKNOWN ? first->wrap_rows : + h->exact_rows[line_first(h, id) & h->ring_mask]; + } + uint64_t cells = term_hist_line_cells(h, id); + if (cols == 1) + cells -= term_hist_line_wide(h, id); + return rows_narrow(cells, cols); +} + +void term_hist_refine_rows(struct hist *h, uint64_t id, int cols, int rows) +{ + if (cols != h->wrap_cols || id >= h->rows_id || id < h->base_id) + return; + uint64_t seq = line_first(h, id); + uint64_t old = term_hist_line_estimate(h, id, cols); + if (rows >= ROWS_UNKNOWN) { + if (!h->exact_rows) { + h->exact_rows = calloc((size_t)h->ring_mask + 1, sizeof(*h->exact_rows)); + if (!h->exact_rows) + return; + } + h->exact_rows[seq & h->ring_mask] = (uint32_t)rows; + } + struct hline *first = part(h, seq); + first->flags |= HLINE_ROWS_EXACT; + first->wrap_rows = row_count_byte((uint64_t)rows); + h->counted_rows = h->counted_rows - old + (uint64_t)rows; +} + +/* Fill descriptor counts only. This never reads the encoded text. */ +void term_hist_prepare_rows(struct hist *h, int cols) +{ + if (cols != h->wrap_cols) + term_hist_set_width(h, cols); + uint64_t stop = hist_end_id(h) - (hist_is_open(h) ? 1 : 0); + for (; h->rows_id < stop; h->rows_id++) { + uint64_t rows = term_hist_line_estimate(h, h->rows_id, cols); + struct hline *first = part(h, line_first(h, h->rows_id)); + first->wrap_rows = row_count_byte(rows); + first->flags &= (uint8_t)~HLINE_ROWS_EXACT; + if (rows < ROWS_UNKNOWN && + (!(term_hist_line_flags(h, h->rows_id) & HLINE_HAS_WIDE) || cols == 1)) + first->flags |= HLINE_ROWS_EXACT; + h->counted_rows += rows; + } +} + +uint64_t term_hist_estimated_rows(const struct hist *h, int cols) +{ + if (!h || cols <= 0 || !h->lines) + return 0; + uint64_t total = cols == h->wrap_cols ? h->counted_rows : 0; + uint64_t first = cols == h->wrap_cols ? h->rows_id : h->base_id; + for (uint64_t id = first; id < hist_end_id(h); id++) + total += term_hist_line_estimate(h, id, cols); + return total; +} + +uint64_t term_hist_rows(const struct hist *h, int cols) +{ + if (!h || cols <= 0) + return 0; + uint64_t total = 0; + for (uint64_t id = hist_first_id(h); id < hist_end_id(h); id++) + total += (uint64_t)term_hist_line_rows(h, id, cols); + return total; +} + +bool term_hist_find_row(const struct hist *h, int target_row, int cols, + uint64_t *out_id, int *out_seg) +{ + if (!h || target_row < 0 || cols <= 0) + return false; + uint64_t total = term_hist_rows(h, cols); + if ((uint64_t)target_row >= total) + return false; + uint64_t id, start; + if ((uint64_t)target_row >= total / 2) { + start = total; + id = hist_end_id(h); + do { + id--; + start -= (uint64_t)term_hist_line_rows(h, id, cols); + } while (start > (uint64_t)target_row && id > h->base_id); + } else { + start = 0; + id = h->base_id; + for (;;) { + uint64_t rows = (uint64_t)term_hist_line_rows(h, id, cols); + if ((uint64_t)target_row < start + rows) + break; + start += rows; + id++; + } + } + *out_id = id; + *out_seg = target_row - (int)start; + return true; +} + +/* Locate a row boundary by its cell offset. Wide wrap pads do not count. */ +uint64_t term_hist_segment_cell(const struct hist *h, uint64_t id, int seg, int cols) +{ + if (!line_valid(h, id)) + return 0; + uint64_t base = id == h->base_cell_id ? h->base_cell : 0; + if (seg <= 0) + return base; + if (!(term_hist_line_flags(h, id) & (HLINE_HAS_WIDE | HLINE_HAS_CLUSTER))) + return base + (uint64_t)seg * cols; + uint64_t first = line_first(h, id), end = line_end(h, first), cells = 0; + int row = 0, col = 0; + for (uint64_t seq = first; seq < end; seq++) { + const struct hline *p = part(h, seq); + if (!p->text_len) + continue; + const uint8_t *t = part_text(h, p), *stop = t + p->text_len; + while (t < stop) { + int width = text_width(t) == 2 && cols > 1 ? 2 : 1; + if (col == cols || col + width > cols) { + if (++row == seg) + return base + cells; + col = 0; + } + cells += width; + col += width; + t = text_skip(t, stop); + } + } + return base + cells; +} + +int term_hist_cell_segment(const struct hist *h, uint64_t id, uint64_t cell, int cols) +{ + if (!line_valid(h, id) || cols <= 0) + return 0; + uint64_t base = id == h->base_cell_id ? h->base_cell : 0; + cell = cell > base ? cell - base : 0; + if (!(term_hist_line_flags(h, id) & (HLINE_HAS_WIDE | HLINE_HAS_CLUSTER))) { + uint64_t cells = term_hist_line_cells(h, id); + if (cells && cell >= cells) + cell = cells - 1; + return (int)(cell / cols); + } + uint64_t first = line_first(h, id), end = line_end(h, first), cells = 0; + int row = 0, col = 0; + for (uint64_t seq = first; seq < end; seq++) { + const struct hline *p = part(h, seq); + if (!p->text_len) + continue; + const uint8_t *t = part_text(h, p), *stop = t + p->text_len; + while (t < stop) { + int width = text_width(t) == 2 && cols > 1 ? 2 : 1; + if (col == cols || col + width > cols) { + row++; + col = 0; + } + if (cells + (uint64_t)width > cell) + return row; + cells += width; + col += width; + t = text_skip(t, stop); + } + } + return row; +} + +/* ---- Life cycle ---- */ + +bool term_hist_init(struct hist *h, int line_limit, size_t byte_cap) +{ + memset(h, 0, sizeof(*h)); + h->all_blank = true; + runs_reset(&h->runs); + if (line_limit <= 0) + return true; + if (byte_cap == 0) { + size_t scaled = (size_t)line_limit * 800; + byte_cap = scaled > 8u * 1024 * 1024 ? scaled : 8u * 1024 * 1024; + } + h->line_limit = (uint32_t)line_limit; + h->max_pages = (uint32_t)(byte_cap / HIST_PAGE_SIZE); + if (h->max_pages == 0) + h->max_pages = 1; + if (h->max_pages > 65535) + h->max_pages = 65535; + h->pages = calloc(h->max_pages, sizeof(*h->pages)); + h->page_used = calloc(h->max_pages, sizeof(*h->page_used)); + h->page_parts = calloc(h->max_pages, sizeof(*h->page_parts)); + h->page_rows = calloc(h->max_pages, sizeof(*h->page_rows)); + h->page_simple = calloc(h->max_pages, sizeof(*h->page_simple)); + h->free_pages = malloc(h->max_pages * sizeof(*h->free_pages)); + if (!h->pages || !h->page_used || !h->page_parts || !h->page_rows || !h->page_simple || !h->free_pages || !ring_reserve(h)) { + term_hist_free(h); + return false; + } + return true; +} + +void term_hist_free(struct hist *h) +{ + if (h->pages) + for (uint32_t i = 0; i < h->npages; i++) + term_aligned_free(h->pages[i]); + free(h->pages); + free(h->page_used); + free(h->page_parts); + free(h->page_rows); + free(h->page_simple); + free(h->free_pages); + term_aligned_free(h->ring); + free(h->blocks); + free(h->exact_rows); + free(h->cont); + free(h->runs.data); + memset(h, 0, sizeof(*h)); +} + +void term_hist_clear(struct hist *h) +{ + h->index_epoch++; + uint64_t end = hist_end_id(h); + h->tail = h->head = 0; + h->base_id = end; + h->base_cell = 0; + h->last_cells = h->last_wide = 0; + h->lines = 0; + h->cont_count = 0; + h->nfree = 0; + for (uint32_t pg = 0; pg < h->npages; pg++) { + h->page_used[pg] = 0; + h->page_parts[pg] = 0; + if (pg != h->cur_page) + h->free_pages[h->nfree++] = (uint16_t)pg; + } + h->all_blank = true; + h->rows_id = end; + h->counted_rows = 0; + h->walk_on = false; + runs_reset(&h->runs); +} diff --git a/src/hist.h b/src/hist.h new file mode 100644 index 0000000..1aceae7 --- /dev/null +++ b/src/hist.h @@ -0,0 +1,321 @@ +#ifndef WSAP_TERM_HIST_H +#define WSAP_TERM_HIST_H + +#include "cell.h" +#include "gstore.h" +#include "compat.h" +#include "pen.h" +#include <stdbool.h> +#include <stddef.h> +#include <stdint.h> +#include <string.h> + +/* + * History of logical lines. A line is stored once, independent of the + * terminal width. A line id counts logical lines: a line that needs more than + * one storage part keeps one id. Ids never change while the line is retained. + */ + +#define HIST_PAGE_SIZE 65536 +#define HIST_PAGE_SLACK 64 +#define HIST_PAGE_USABLE (HIST_PAGE_SIZE - HIST_PAGE_SLACK) +#define HIST_MAX_PART_CELLS 65535 + +enum hline_flags { + HLINE_NARROW = 1 << 0, /* one ASCII byte per cell */ + HLINE_HAS_WIDE = 1 << 1, + HLINE_HAS_CLUSTER = 1 << 2, + HLINE_HAS_LINK = 1 << 3, + HLINE_CONT = 1 << 4, /* continues the line of the previous part */ + HLINE_ROWS_EXACT = 1 << 6, /* first part: cached row count is exact */ + HLINE_OPEN = 1 << 5, /* the part that grows; only the last part */ +}; + +/* One storage part. The first part of a line holds the line's row count. */ +struct hline { + uint32_t off; /* page << 16 | offset in the page */ + uint32_t text_len; /* bytes */ + uint16_t cells; /* display width */ + uint16_t wide; /* wide glyphs; a one-column view clips them */ + uint16_t attr_len; /* 0: every cell has the default pen */ + uint8_t flags; + uint8_t wrap_rows; /* first part: rows of the line at wrap_cols; 255: count them */ +}; + +_Static_assert(sizeof(struct hline) == 16, "struct hline must stay 16 bytes"); + +/* Attribute runs of the open part. They move to the page when it closes. + * count comes before the pens, so a wide store of the pens never covers + * it. */ +struct hist_runs { + uint8_t *data; + uint32_t len, cap; + uint32_t count; /* cells of the pending run */ + struct pen_attr base; /* pen of the last emitted run */ + struct pen_attr pen; /* pen of the pending run */ +}; + +/* Totals for 16 closed lines in one page. A sequence and epoch validate it. */ +struct hist_block { + uint64_t seq, epoch; + uint32_t page, refs, rows; + uint16_t cols; +}; + +struct hist_ascii_line { + const char *ptr; + uint16_t len; + uint8_t wrapped; +}; + +/* + * One row of text in the history format, built by appending. A screen text + * row holds one, so a row that scrolls into history moves by copy. The runs + * start from the pen start; their buffer never grows, because every run + * covers at least one cell and the caller sizes it for a full row. + */ +struct text_row { + uint8_t *text; + uint8_t *storage; /* owned buffer, when text uses compact ASCII storage */ + uint16_t text_len; + uint16_t cells; + uint16_t wide; + uint16_t fill_to; /* blanks with pen fill follow the text up to here */ + uint8_t flags; /* HLINE_NARROW, HLINE_HAS_WIDE, HLINE_HAS_LINK */ + struct pen_attr start; + struct pen_attr fill; + struct hist_runs runs; +}; + +/* Bytes a text row of cols cells may need: text, and runs. */ +#define TEXT_ROW_BYTES(cols) ((size_t)(cols) * 4 + 16) +#define TEXT_ROW_RUNS(cols) (((size_t)(cols) + 2) * 22) + +/* A pen as three words: fg, bg, and flags | link << 16 (the layout of + * pen_attr.flags and .link). The hot path reads and writes whole words, so + * a store is never read back through a wider load. */ +static inline uint32_t pen_sl(const struct pen_attr *p) +{ + uint32_t sl; + memcpy(&sl, &p->flags, 4); + return sl; +} + +static inline void pen_set(struct pen_attr *p, uint32_t fg, uint32_t bg, uint32_t sl) +{ + p->fg = fg; + p->bg = bg; + memcpy(&p->flags, &sl, 4); +} + +/* Emit the pending run in the general form. */ +void term_hist_runs_emit_slow(struct hist_runs *r); + +/* Close the pending run and start one with the pen fg, bg, sl. The caller + * sized the buffer. A run that changes only colors and covers fewer than + * 128 cells is a header byte (bit 0 fg, bit 1 bg), a count byte, and the + * changed colors. */ +static inline void hist_runs_switch(struct hist_runs *r, uint32_t fg, uint32_t bg, uint32_t sl) +{ + if (r->count) { + uint32_t psl = pen_sl(&r->pen); + if (TERM_EXPECT(r->count < 0x80 && psl == pen_sl(&r->base), 1)) { + uint8_t *p = r->data + r->len; + uint32_t dfg = r->pen.fg != r->base.fg, dbg = r->pen.bg != r->base.bg; + p[0] = (uint8_t)(dfg | dbg << 1); + p[1] = (uint8_t)r->count; + memcpy(p + 2, &r->pen.fg, 4); + p += 2 + dfg * 4; + memcpy(p, &r->pen.bg, 4); + r->len = (uint32_t)(p + dbg * 4 - r->data); + } else { + term_hist_runs_emit_slow(r); + } + pen_set(&r->base, r->pen.fg, r->pen.bg, psl); + } + pen_set(&r->pen, fg, bg, sl); + r->count = 0; +} + +static inline void hist_runs_add_pen(struct hist_runs *r, uint32_t fg, uint32_t bg, uint32_t sl, + uint32_t n) +{ + if (fg != r->pen.fg || bg != r->pen.bg || sl != pen_sl(&r->pen)) + hist_runs_switch(r, fg, bg, sl); + r->count += n; +} + +static inline void hist_runs_add(struct hist_runs *r, const struct pen_attr *pen, uint32_t n) +{ + hist_runs_add_pen(r, pen->fg, pen->bg, pen_sl(pen), n); +} + +/* Start a text row. Each field gets its own store of its own width, because + * the first append reads them back at once. */ +static inline void text_row_start(struct text_row *row, const struct pen_attr *start) +{ + if (row->storage) row->text = row->storage; + uint32_t fg = start->fg, bg = start->bg, sl = pen_sl(start); + row->text_len = 0; + row->cells = 0; + row->wide = 0; + row->fill_to = 0; + row->flags = HLINE_NARROW; + pen_set(&row->start, fg, bg, sl); + row->runs.len = 0; + row->runs.count = 0; + pen_set(&row->runs.base, fg, bg, sl); + pen_set(&row->runs.pen, fg, bg, sl); +} + +static inline void text_row_unshare(struct text_row *row) +{ + if (row->storage && row->text != row->storage) { + memcpy(row->storage, row->text, row->text_len); + row->text = row->storage; + } +} + +/* The pen of the last cell of a text row. */ +static inline struct pen_attr text_row_last_pen(const struct text_row *row) +{ + return row->runs.pen; +} + +/* Write the cells of a text row to out; return the cell count. */ +int term_hist_text_decode(const struct text_row *row, struct cell *out, int cap, struct gstore *g); +void term_hist_text_mark_links(const struct text_row *row, void (*mark)(void *ud, uint16_t link), void *ud); + +struct hist_dec_cache { + const uint8_t *p, *end; + struct pen_attr pending_pen; + struct pen_attr pen; + uint64_t epoch; + uint64_t id; + uint64_t seq; + uint32_t seg; + uint32_t off; + uint32_t col; + uint32_t pending_count; + uint32_t left; + uint16_t cols; + bool pending; + bool valid; +}; + +struct hist { + struct hline *ring; + struct hist_block *blocks; + uint64_t tail, head; /* part sequence numbers [tail, head) */ + uint64_t base_id; /* line id of the part at tail */ + uint64_t *cont; /* CONT parts after tail, oldest first */ + uint8_t **pages; + uint32_t *page_used; + uint32_t *page_parts; + uint32_t *page_rows; + uint8_t *page_simple; + uint16_t *free_pages; + uint32_t *exact_rows; /* counts that exceed the descriptor byte */ + struct hist_runs runs; + struct hist_dec_cache dec; + + /* Rows at wrap_cols. counted_rows covers [base_id, rows_id). + * Snapshot capture counts later closed lines from descriptors. + * A wide open line keeps a walk that resumes where it stopped. */ + uint64_t rows_id; + uint64_t counted_rows; + uint64_t walk_seq; + + /* Changes when retained line ids can name different content. */ + uint64_t index_epoch; + uint64_t base_cell_id, base_cell; /* removed prefix and its line */ + uint64_t last_id, last_cells, last_wide; + + uint32_t ring_mask; + uint32_t lines; + uint32_t line_limit; + uint32_t cont_first, cont_count, cont_mask; + uint32_t npages, max_pages, nfree, cur_page; + uint32_t walk_off, walk_col, walk_rows; + uint16_t wrap_cols; + bool all_blank; + bool walk_on; +}; + +static inline uint32_t rows_narrow(uint64_t cells, int cols) +{ + return cells <= (uint64_t)cols ? 1 : + (uint32_t)((cells + (uint64_t)cols - 1) / (uint64_t)cols); +} + +bool term_hist_init(struct hist *h, int line_limit, size_t byte_cap); +void term_hist_free(struct hist *h); +void term_hist_clear(struct hist *h); + +/* Push one screen row. It joins the open line, or starts a new line. A + * wrapped row leaves the line open for the next row. */ +void term_hist_push_row(struct hist *h, const struct cell *cells, int len, bool wrapped, + const struct gstore *g); +void term_hist_push_ascii(struct hist *h, const char *bytes, int n, + const struct pen_attr *pen, bool wrapped); +void term_hist_push_ascii_lines(struct hist *h, const struct hist_ascii_line *lines, + int count, const struct pen_attr *pen); +/* Ends include LF. ends[0] is the start of the first line. */ +void term_hist_push_ascii_lf(struct hist *h, const char *bytes, const uint16_t *ends, int count); +/* Push a text row as one screen row, like term_hist_push_row(). */ +void term_hist_push_text(struct hist *h, const struct text_row *row, bool wrapped); +/* Push n empty unwrapped rows. The first one closes an open line. */ +void term_hist_push_blank(struct hist *h, size_t n); +void term_hist_close_open_line(struct hist *h); + +/* Remove the last line. Write its cells to out (at most cap) and return the + * number of cells in the line. */ +int term_hist_pop_line(struct hist *h, struct cell *out, int cap, struct gstore *g); +/* Copy the last line's cells to out, as term_hist_pop_line() does, and keep the line. + * Return the number of cells in the line. */ +int term_hist_peek_last_line(const struct hist *h, struct cell *out, int cap, struct gstore *g); +/* Take up to n cells back off the end of the last line. Return the count. */ +int term_hist_pop_tail_cells(struct hist *h, int n, struct cell *out, struct gstore *g); + +static inline uint64_t hist_first_id(const struct hist *h) +{ + return h->base_id; +} + +static inline uint64_t hist_end_id(const struct hist *h) +{ + return h->base_id + h->lines; +} + +static inline bool hist_is_open(const struct hist *h) +{ + return h->head != h->tail && (h->ring[(h->head - 1) & h->ring_mask].flags & HLINE_OPEN); +} + +uint64_t term_hist_line_cells(const struct hist *h, uint64_t id); +uint64_t term_hist_line_wide(const struct hist *h, uint64_t id); +uint8_t term_hist_line_flags(const struct hist *h, uint64_t id); +int term_hist_line_rows(const struct hist *h, uint64_t id, int cols); +int term_hist_decode_segment(const struct hist *h, uint64_t id, int seg, int cols, + struct cell *out, struct gstore *g); +int term_hist_decode_segments(const struct hist *h, uint64_t id, int start_seg, int count, int cols, + struct cell *out, int *lens, struct gstore *g); + +uint64_t term_hist_segment_cell(const struct hist *h, uint64_t id, int seg, int cols); +int term_hist_cell_segment(const struct hist *h, uint64_t id, uint64_t cell, int cols); + +void term_hist_mark_links(const struct hist *h, void (*mark)(void *ud, uint16_t link), void *ud); + +/* Set the width that pushed rows have. It costs O(1): the stored row counts + * become stale. Reads count them without changing the history. */ +void term_hist_set_width(struct hist *h, int cols); + +/* Rows of all retained lines at cols. */ +uint64_t term_hist_rows(const struct hist *h, int cols); +uint64_t term_hist_line_estimate(const struct hist *h, uint64_t id, int cols); +uint64_t term_hist_estimated_rows(const struct hist *h, int cols); +void term_hist_prepare_rows(struct hist *h, int cols); +void term_hist_refine_rows(struct hist *h, uint64_t id, int cols, int rows); +bool term_hist_find_row(const struct hist *h, int target_row, int cols, uint64_t *out_id, int *out_seg); + +#endif diff --git a/src/hist_view.c b/src/hist_view.c new file mode 100644 index 0000000..d413c43 --- /dev/null +++ b/src/hist_view.c @@ -0,0 +1,366 @@ +#include "hist_view.h" +#include <limits.h> +#include <stdlib.h> +#include <string.h> + +void term_hist_view_init(struct hist_view *v) +{ + *v = (struct hist_view){0}; +} + +void term_hist_view_free(struct hist_view *v) +{ + if (!v) + return; + free(v->fenwick); + free(v->counts); + *v = (struct hist_view){0}; +} + +void term_hist_view_clear(struct hist_view *v) +{ + v->anchor = (struct view_anchor){0}; + v->index_cols = 0; +} + +void term_hist_view_reset_bottom(struct hist_view *v) +{ + v->anchor = (struct view_anchor){0}; +} + +static uint64_t prefix(const struct hist_view *v, uint32_t n) +{ + uint64_t sum = 0; + for (; n; n &= n - 1) + sum += v->fenwick[n]; + return sum; +} + +static void set_count(struct hist_view *v, uint64_t id, uint32_t count) +{ + uint32_t slot = (uint32_t)id & (v->cap - 1); + int64_t delta = (int64_t)count - v->counts[slot]; + v->counts[slot] = count; + v->total_rows = (uint64_t)((int64_t)v->total_rows + delta); + for (uint32_t i = slot + 1; i <= v->cap; i += i & -i) + v->fenwick[i] = (uint64_t)((int64_t)v->fenwick[i] + delta); +} + +static bool current(const struct hist_view *v, const struct hist *h, int cols) +{ + return v->index_cols == cols && v->epoch == h->index_epoch && + v->first_id == hist_first_id(h) && v->end_id == hist_end_id(h) && + v->head == h->head && v->tail == h->tail && + v->last_cells == (h->lines ? term_hist_line_cells(h, hist_end_id(h) - 1) : 0); +} + +bool term_hist_view_sync(struct hist_view *v, const struct hist *h, int cols) +{ + if (current(v, h, cols)) + return true; + uint64_t first = hist_first_id(h), end = hist_end_id(h); + bool rebuild = v->index_cols != cols || v->epoch != h->index_epoch || + first < v->first_id || first >= v->end_id || end < v->end_id; + if (!rebuild) { + uint64_t changed = first - v->first_id + end - v->end_id; + rebuild = changed > v->cap / 4; + } + if (h->lines > v->cap || !v->cap) { + uint32_t cap = 16; + while (cap < h->lines) { + if (cap >= UINT32_MAX / 2) + return false; + cap *= 2; + } + uint64_t *tree = calloc((size_t)cap + 1, sizeof(*tree)); + uint32_t *counts = calloc(cap, sizeof(*counts)); + if (!tree || !counts) { + free(tree); + free(counts); + return false; + } + free(v->fenwick); + free(v->counts); + v->fenwick = tree; + v->counts = counts; + v->cap = cap; + rebuild = true; + } + if (rebuild) { + memset(v->counts, 0, (size_t)v->cap * sizeof(*v->counts)); + memset(v->fenwick, 0, ((size_t)v->cap + 1) * sizeof(*v->fenwick)); + for (uint64_t id = first; id < end; id++) + v->counts[(uint32_t)id & (v->cap - 1)] = (uint32_t)term_hist_line_estimate(h, id, cols); + v->total_rows = 0; + /* Build all tree nodes in one pass. */ + for (uint32_t i = 1; i <= v->cap; i++) { + v->fenwick[i] += v->counts[i - 1]; + v->total_rows += v->counts[i - 1]; + uint32_t parent = i + (i & -i); + if (parent <= v->cap) + v->fenwick[parent] += v->fenwick[i]; + } + } else { + for (uint64_t id = v->first_id; id < first; id++) + set_count(v, id, 0); + /* Only the old final line can grow or close. */ + if (v->end_id > first && term_hist_line_cells(h, v->end_id - 1) != v->last_cells) + set_count(v, v->end_id - 1, (uint32_t)term_hist_line_estimate(h, v->end_id - 1, cols)); + for (uint64_t id = v->end_id; id < end; id++) + set_count(v, id, (uint32_t)term_hist_line_estimate(h, id, cols)); + } + v->first_id = first; + v->end_id = end; + v->index_cols = cols; + v->epoch = h->index_epoch; + v->head = h->head; + v->tail = h->tail; + v->last_cells = h->lines ? term_hist_line_cells(h, end - 1) : 0; + return true; +} + +/* The anchor line is a visited line, even before the next capture. */ +static uint64_t anchor_extra(const struct hist_view *v, const struct hist *h, int cols) +{ + uint64_t id = v->anchor.line_id; + if (!v->anchor.active || id < hist_first_id(h) || id >= hist_end_id(h)) + return 0; + uint64_t estimated = current(v, h, cols) ? v->counts[(uint32_t)id & (v->cap - 1)] : + term_hist_line_estimate(h, id, cols); + uint64_t exact = (uint64_t)term_hist_line_rows(h, id, cols); + return exact > estimated ? exact - estimated : 0; +} + +uint64_t term_hist_view_rows(const struct hist_view *v, const struct hist *h, int cols) +{ + uint64_t rows = current(v, h, cols) ? v->total_rows : term_hist_estimated_rows(h, cols); + return rows + anchor_extra(v, h, cols); +} + +/* Rows of the retained lines before id, in logical order. */ +static uint64_t before(const struct hist_view *v, uint64_t id) +{ + uint32_t start = (uint32_t)v->first_id & (v->cap - 1); + uint64_t n = id - v->first_id; + uint32_t stop = start + (uint32_t)n; + if (stop <= v->cap) + return prefix(v, stop) - prefix(v, start); + return prefix(v, v->cap) - prefix(v, start) + prefix(v, stop - v->cap); +} + +/* Physical slot of a row in tree order. */ +static uint32_t select_slot(const struct hist_view *v, uint64_t row) +{ + uint32_t slot = 0; + for (uint32_t bit = v->cap; bit; bit >>= 1) { + uint32_t next = slot + bit; + if (next <= v->cap && v->fenwick[next] <= row) { + row -= v->fenwick[next]; + slot = next; + } + } + return slot; +} + +bool term_hist_view_find_row(const struct hist_view *v, const struct hist *h, int row, + int cols, uint64_t *id, int *seg) +{ + if (!current(v, h, cols)) { + if (row < 0) + return false; + uint64_t target = (uint64_t)row; + for (uint64_t line = hist_first_id(h); line < hist_end_id(h); line++) { + uint64_t count = term_hist_line_estimate(h, line, cols); + if (target < count) { + *id = line; + *seg = (int)target; + return true; + } + target -= count; + } + return false; + } + uint64_t total = v->total_rows; + if (row < 0 || (uint64_t)row >= total) + return false; + uint32_t start = (uint32_t)v->first_id & (v->cap - 1); + uint64_t low = prefix(v, start), high = total - low; + uint64_t physical = (uint64_t)row < high ? low + row : (uint64_t)row - high; + uint32_t slot = select_slot(v, physical); + *id = v->first_id + ((slot - start) & (v->cap - 1)); + *seg = (int)(physical - prefix(v, slot)); + return true; +} + +static int rows_as_int(uint64_t rows) +{ + return rows > INT_MAX ? INT_MAX : (int)rows; +} + +int term_hist_view_scroll_offset(const struct hist_view *v, const struct hist *h, int cols) +{ + if (!v->anchor.active || !h->lines) + return 0; + uint64_t id = v->anchor.line_id; + if (id < hist_first_id(h)) + return rows_as_int(term_hist_view_rows(v, h, cols)); + if (id >= hist_end_id(h)) + return 0; + int seg = v->cols == cols ? (int)v->anchor.seg : + term_hist_cell_segment(h, id, v->anchor.cell, cols); + uint64_t rows = 0; + if (current(v, h, cols)) + rows = v->total_rows - before(v, id); + else + for (; id < hist_end_id(h); id++) + rows += term_hist_line_estimate(h, id, cols); + rows += anchor_extra(v, h, cols); + return rows > (uint64_t)seg ? rows_as_int(rows - seg) : 0; +} + +void term_hist_view_note_push(struct hist_view *v, const struct hist *h, int cols) +{ + if (!v->anchor.active) + return; + if (v->anchor.line_id < hist_first_id(h)) + v->anchor = (struct view_anchor){ .line_id = hist_first_id(h), .active = true }; + uint64_t base = term_hist_segment_cell(h, v->anchor.line_id, 0, cols); + if (v->anchor.base_cell != base) { + if (v->anchor.cell < base) + v->anchor.cell = base; + v->anchor.seg = (uint32_t)term_hist_cell_segment(h, v->anchor.line_id, v->anchor.cell, cols); + v->anchor.base_cell = base; + } +} + +void term_hist_view_rewrap(struct hist_view *v, const struct hist *h, int cols) +{ + term_hist_view_note_push(v, h, cols); + if (v->anchor.active) { + if (v->anchor.line_id >= hist_end_id(h)) + term_hist_view_reset_bottom(v); + else + v->anchor.seg = (uint32_t)term_hist_cell_segment(h, v->anchor.line_id, v->anchor.cell, cols); + } + v->cols = cols; + v->index_cols = 0; +} + +void term_hist_view_scroll_to(struct hist_view *v, const struct hist *h, int cols, + struct line_pos screen_top, int offset) +{ + (void)screen_top; + if (offset <= 0 || cols <= 0 || !h->lines) { + term_hist_view_reset_bottom(v); + return; + } + /* Allocation failure uses the read-only history walk. */ + uint64_t total = term_hist_view_rows(v, h, cols); + if ((uint64_t)offset > total) + offset = rows_as_int(total); + uint64_t id; + int seg; + uint64_t row = total - offset; + if (row > INT_MAX || !term_hist_view_find_row(v, h, (int)row, cols, &id, &seg)) + return; + v->anchor = (struct view_anchor){ .line_id = id, .seg = (uint32_t)seg, + .cell = term_hist_segment_cell(h, id, seg, cols), + .base_cell = term_hist_segment_cell(h, id, 0, cols), .active = true }; + v->cols = cols; +} + +/* Relative movement visits only the logical lines it crosses. */ +void term_hist_view_scroll(struct hist_view *v, const struct hist *h, int cols, + struct line_pos screen_top, int lines) +{ + (void)screen_top; + if (!lines || cols <= 0 || !h->lines) + return; + uint64_t id; + int64_t seg; + if (v->anchor.active) { + id = v->anchor.line_id; + if (id < hist_first_id(h)) + id = hist_first_id(h); + if (id >= hist_end_id(h)) + id = hist_end_id(h) - 1; + seg = v->cols == cols ? (int64_t)v->anchor.seg : term_hist_cell_segment(h, id, v->anchor.cell, cols); + } else { + if (lines < 0) + return; + id = hist_end_id(h) - 1; + seg = term_hist_line_rows(h, id, cols); + } + seg -= (int64_t)lines; + while (seg < 0 && id > hist_first_id(h)) + seg += term_hist_line_rows(h, --id, cols); + if (seg < 0) + seg = 0; + for (;;) { + int count = term_hist_line_rows(h, id, cols); + if (seg < count) + break; + seg -= count; + if (++id == hist_end_id(h)) { + term_hist_view_reset_bottom(v); + return; + } + } + v->anchor = (struct view_anchor){ .line_id = id, .seg = (uint32_t)seg, + .cell = term_hist_segment_cell(h, id, (int)seg, cols), + .base_cell = term_hist_segment_cell(h, id, 0, cols), .active = true }; + v->cols = cols; +} + +/* A viewport walks exact segments from its anchor, never an estimated row. */ +bool term_hist_view_row_pos(const struct hist_view *v, const struct hist *h, int cols, + int screen_rows, struct line_pos screen_top, int y, + uint64_t *out_id, int *out_seg, int *out_screen_y) +{ + (void)screen_top; + if (v->anchor.active && h->lines) { + uint64_t id = v->anchor.line_id; + int seg = v->cols == cols ? (int)v->anchor.seg : + term_hist_cell_segment(h, id, v->anchor.cell, cols); + if (id < hist_first_id(h)) { + id = hist_first_id(h); + seg = 0; + } + int64_t target = (int64_t)seg + y; + while (id < hist_end_id(h)) { + int count = term_hist_line_rows(h, id, cols); + if (target < count) { + *out_id = id; + *out_seg = (int)target; + return true; + } + target -= count; + id++; + } + *out_screen_y = target < screen_rows ? (int)target : screen_rows - 1; + return false; + } + *out_screen_y = y; + return false; +} + +/* Wide line counts become exact when the viewport visits them. */ +void term_hist_view_refine(struct hist_view *v, struct hist *h, int cols, int rows) +{ + if (!v->anchor.active || !current(v, h, cols)) + return; + uint64_t id = v->anchor.line_id; + int remaining = rows; + int seg = (int)v->anchor.seg; + if (id < hist_first_id(h)) { + id = hist_first_id(h); + seg = 0; + } + for (; id < hist_end_id(h) && remaining > 0; id++) { + int count = term_hist_line_rows(h, id, cols); + set_count(v, id, (uint32_t)count); + term_hist_refine_rows(h, id, cols, count); + remaining -= count - seg; + seg = 0; + } +} diff --git a/src/hist_view.h b/src/hist_view.h new file mode 100644 index 0000000..0214c0b --- /dev/null +++ b/src/hist_view.h @@ -0,0 +1,60 @@ +#ifndef TERM_HIST_VIEW_H +#define TERM_HIST_VIEW_H + +#include "hist.h" + +struct line_pos { + uint64_t id; + uint32_t seg; +}; + +struct view_anchor { + uint64_t line_id; + uint64_t cell; + uint64_t base_cell; + uint32_t seg; + bool active; +}; + +struct hist_view { + struct view_anchor anchor; + + /* Line ids map to circular slots. Tree sums use physical slot order. */ + uint64_t *fenwick; + uint32_t *counts; + uint64_t total_rows; + uint64_t first_id, end_id; + uint64_t epoch, head, tail, last_cells; + int cols; + uint32_t cap; + int index_cols; +}; + +void term_hist_view_init(struct hist_view *v); +void term_hist_view_free(struct hist_view *v); +void term_hist_view_clear(struct hist_view *v); + +/* Update the index during snapshot capture. Reads do not change it. */ +bool term_hist_view_sync(struct hist_view *v, const struct hist *h, int cols); +void term_hist_view_refine(struct hist_view *v, struct hist *h, int cols, int rows); +uint64_t term_hist_view_rows(const struct hist_view *v, const struct hist *h, int cols); +bool term_hist_view_find_row(const struct hist_view *v, const struct hist *h, int row, + int cols, uint64_t *id, int *seg); + +/* Read the offset from the anchor at the current width. */ +int term_hist_view_scroll_offset(const struct hist_view *v, const struct hist *h, int cols); +void term_hist_view_scroll(struct hist_view *v, const struct hist *h, int cols, + struct line_pos screen_top, int lines); +void term_hist_view_scroll_to(struct hist_view *v, const struct hist *h, int cols, + struct line_pos screen_top, int offset); +void term_hist_view_reset_bottom(struct hist_view *v); + +/* Clamp an evicted anchor. This does not update the index on the write path. */ +void term_hist_view_note_push(struct hist_view *v, const struct hist *h, int cols); +void term_hist_view_rewrap(struct hist_view *v, const struct hist *h, int cols); + +bool term_hist_view_row_pos(const struct hist_view *v, const struct hist *h, int cols, + int screen_rows, struct line_pos screen_top, int y, + uint64_t *out_id, int *out_seg, int *out_screen_y); + +#endif diff --git a/src/input.c b/src/input.c new file mode 100644 index 0000000..49e41b2 --- /dev/null +++ b/src/input.c @@ -0,0 +1,771 @@ +#include "term.h" +#include "unicode/utf8.h" + +#include <string.h> + +struct seqbuf { + char *p; + size_t left; + bool overflow; +}; + +static inline void sb_init(struct seqbuf *sb, char *buf, size_t buf_sz) +{ + sb->p = buf; + sb->left = buf_sz; + sb->overflow = false; +} + +static inline void sb_ch(struct seqbuf *sb, char c) +{ + if (!sb->left) { sb->overflow = true; return; } + *sb->p++ = c; + sb->left--; +} + +static inline void sb_u32(struct seqbuf *sb, uint32_t val) +{ + char tmp[10]; + size_t n = 0; + do { + tmp[n++] = (char)('0' + (val % 10)); + val /= 10; + } while (val); + while (n) sb_ch(sb, tmp[--n]); +} + +static inline void sb_str(struct seqbuf *sb, const char *s, size_t len) +{ + for (size_t i = 0; i < len; i++) sb_ch(sb, s[i]); +} + +static inline size_t sb_len(const struct seqbuf *sb, const char *buf) +{ + return (size_t)(sb->p - buf); +} + +static inline void sb_bytes(struct seqbuf *sb, const char *s, size_t len) +{ + if (len > sb->left) { sb->overflow = true; return; } + memcpy(sb->p, s, len); + sb->p += len; + sb->left -= len; +} + +struct kitty_desc { + uint32_t code; + char final; +}; + +static const struct kitty_desc kitty_desc[TERM_KEY_COUNT] = { + [TERM_KEY_RETURN] = { 13, 'u' }, + [TERM_KEY_BACKSPACE] = { 127, 'u' }, + [TERM_KEY_TAB] = { 9, 'u' }, + [TERM_KEY_ESCAPE] = { 27, 'u' }, + + [TERM_KEY_UP] = { 1, 'A' }, + [TERM_KEY_DOWN] = { 1, 'B' }, + [TERM_KEY_RIGHT] = { 1, 'C' }, + [TERM_KEY_LEFT] = { 1, 'D' }, + [TERM_KEY_HOME] = { 1, 'H' }, + [TERM_KEY_END] = { 1, 'F' }, + [TERM_KEY_PAGE_UP] = { 5, '~' }, + [TERM_KEY_PAGE_DOWN] = { 6, '~' }, + [TERM_KEY_INSERT] = { 2, '~' }, + [TERM_KEY_DELETE] = { 3, '~' }, + + [TERM_KEY_F1] = { 1, 'P' }, + [TERM_KEY_F2] = { 1, 'Q' }, + [TERM_KEY_F3] = { 13, '~' }, + [TERM_KEY_F4] = { 1, 'S' }, + [TERM_KEY_F5] = { 15, '~' }, + [TERM_KEY_F6] = { 17, '~' }, + [TERM_KEY_F7] = { 18, '~' }, + [TERM_KEY_F8] = { 19, '~' }, + [TERM_KEY_F9] = { 20, '~' }, + [TERM_KEY_F10] = { 21, '~' }, + [TERM_KEY_F11] = { 23, '~' }, + [TERM_KEY_F12] = { 24, '~' }, + [TERM_KEY_F13] = { 57376, 'u' }, + [TERM_KEY_F14] = { 57377, 'u' }, + [TERM_KEY_F15] = { 57378, 'u' }, + [TERM_KEY_F16] = { 57379, 'u' }, + [TERM_KEY_F17] = { 57380, 'u' }, + [TERM_KEY_F18] = { 57381, 'u' }, + [TERM_KEY_F19] = { 57382, 'u' }, + [TERM_KEY_F20] = { 57383, 'u' }, + [TERM_KEY_F21] = { 57384, 'u' }, + [TERM_KEY_F22] = { 57385, 'u' }, + [TERM_KEY_F23] = { 57386, 'u' }, + [TERM_KEY_F24] = { 57387, 'u' }, + [TERM_KEY_F25] = { 57388, 'u' }, + [TERM_KEY_F26] = { 57389, 'u' }, + [TERM_KEY_F27] = { 57390, 'u' }, + [TERM_KEY_F28] = { 57391, 'u' }, + [TERM_KEY_F29] = { 57392, 'u' }, + [TERM_KEY_F30] = { 57393, 'u' }, + [TERM_KEY_F31] = { 57394, 'u' }, + [TERM_KEY_F32] = { 57395, 'u' }, + [TERM_KEY_F33] = { 57396, 'u' }, + [TERM_KEY_F34] = { 57397, 'u' }, + [TERM_KEY_F35] = { 57398, 'u' }, + + [TERM_KEY_KP_0] = { 57399, 'u' }, + [TERM_KEY_KP_1] = { 57400, 'u' }, + [TERM_KEY_KP_2] = { 57401, 'u' }, + [TERM_KEY_KP_3] = { 57402, 'u' }, + [TERM_KEY_KP_4] = { 57403, 'u' }, + [TERM_KEY_KP_5] = { 57404, 'u' }, + [TERM_KEY_KP_6] = { 57405, 'u' }, + [TERM_KEY_KP_7] = { 57406, 'u' }, + [TERM_KEY_KP_8] = { 57407, 'u' }, + [TERM_KEY_KP_9] = { 57408, 'u' }, + [TERM_KEY_KP_DECIMAL] = { 57409, 'u' }, + [TERM_KEY_KP_DIVIDE] = { 57410, 'u' }, + [TERM_KEY_KP_MULTIPLY] = { 57411, 'u' }, + [TERM_KEY_KP_SUBTRACT] = { 57412, 'u' }, + [TERM_KEY_KP_ADD] = { 57413, 'u' }, + [TERM_KEY_KP_ENTER] = { 57414, 'u' }, + [TERM_KEY_KP_EQUAL] = { 57415, 'u' }, + [TERM_KEY_KP_SEPARATOR] = { 57416, 'u' }, + [TERM_KEY_KP_LEFT] = { 57417, 'u' }, + [TERM_KEY_KP_RIGHT] = { 57418, 'u' }, + [TERM_KEY_KP_UP] = { 57419, 'u' }, + [TERM_KEY_KP_DOWN] = { 57420, 'u' }, + [TERM_KEY_KP_PAGE_UP] = { 57421, 'u' }, + [TERM_KEY_KP_PAGE_DOWN] = { 57422, 'u' }, + [TERM_KEY_KP_HOME] = { 57423, 'u' }, + [TERM_KEY_KP_END] = { 57424, 'u' }, + [TERM_KEY_KP_INSERT] = { 57425, 'u' }, + [TERM_KEY_KP_DELETE] = { 57426, 'u' }, + [TERM_KEY_KP_BEGIN] = { 1, 'E' }, + + [TERM_KEY_CAPS_LOCK] = { 57358, 'u' }, + [TERM_KEY_SCROLL_LOCK] = { 57359, 'u' }, + [TERM_KEY_NUM_LOCK] = { 57360, 'u' }, + [TERM_KEY_PRINT_SCREEN] = { 57361, 'u' }, + [TERM_KEY_PAUSE] = { 57362, 'u' }, + [TERM_KEY_MENU] = { 57363, 'u' }, + + [TERM_KEY_MEDIA_PLAY] = { 57428, 'u' }, + [TERM_KEY_MEDIA_PAUSE] = { 57429, 'u' }, + [TERM_KEY_MEDIA_STOP] = { 57432, 'u' }, + [TERM_KEY_MEDIA_FORWARD] = { 57433, 'u' }, + [TERM_KEY_MEDIA_REWIND] = { 57434, 'u' }, + [TERM_KEY_MEDIA_NEXT] = { 57435, 'u' }, + [TERM_KEY_MEDIA_PREV] = { 57436, 'u' }, + [TERM_KEY_MEDIA_RECORD] = { 57437, 'u' }, + [TERM_KEY_VOLUME_DOWN] = { 57438, 'u' }, + [TERM_KEY_VOLUME_UP] = { 57439, 'u' }, + [TERM_KEY_MUTE] = { 57440, 'u' }, + + [TERM_KEY_LSHIFT] = { 57441, 'u' }, + [TERM_KEY_LCTRL] = { 57442, 'u' }, + [TERM_KEY_LALT] = { 57443, 'u' }, + [TERM_KEY_LSUPER] = { 57444, 'u' }, + [TERM_KEY_LHYPER] = { 57445, 'u' }, + [TERM_KEY_LMETA] = { 57446, 'u' }, + [TERM_KEY_RSHIFT] = { 57447, 'u' }, + [TERM_KEY_RCTRL] = { 57448, 'u' }, + [TERM_KEY_RALT] = { 57449, 'u' }, + [TERM_KEY_RSUPER] = { 57450, 'u' }, + [TERM_KEY_RHYPER] = { 57451, 'u' }, + [TERM_KEY_RMETA] = { 57452, 'u' }, + [TERM_KEY_ISO_LEVEL3_SHIFT] = { 57453, 'u' }, + [TERM_KEY_ISO_LEVEL5_SHIFT] = { 57454, 'u' }, +}; + +struct legacy_desc { + uint8_t tilde; + char final; + bool app_cursor; + char keypad_ss3; + enum term_key equiv; +}; + +static const struct legacy_desc legacy_desc[TERM_KEY_COUNT] = { + [TERM_KEY_UP] = { .tilde = 0, .final = 'A', .app_cursor = true }, + [TERM_KEY_DOWN] = { .tilde = 0, .final = 'B', .app_cursor = true }, + [TERM_KEY_RIGHT] = { .tilde = 0, .final = 'C', .app_cursor = true }, + [TERM_KEY_LEFT] = { .tilde = 0, .final = 'D', .app_cursor = true }, + [TERM_KEY_HOME] = { .tilde = 0, .final = 'H', .app_cursor = true }, + [TERM_KEY_END] = { .tilde = 0, .final = 'F', .app_cursor = true }, + [TERM_KEY_PAGE_UP] = { .tilde = 5, .final = '~' }, + [TERM_KEY_PAGE_DOWN] = { .tilde = 6, .final = '~' }, + [TERM_KEY_INSERT] = { .tilde = 2, .final = '~' }, + [TERM_KEY_DELETE] = { .tilde = 3, .final = '~' }, + + [TERM_KEY_F1] = { .tilde = 0, .final = 'P' }, + [TERM_KEY_F2] = { .tilde = 0, .final = 'Q' }, + [TERM_KEY_F3] = { .tilde = 0, .final = 'R' }, + [TERM_KEY_F4] = { .tilde = 0, .final = 'S' }, + [TERM_KEY_F5] = { .tilde = 15, .final = '~' }, + [TERM_KEY_F6] = { .tilde = 17, .final = '~' }, + [TERM_KEY_F7] = { .tilde = 18, .final = '~' }, + [TERM_KEY_F8] = { .tilde = 19, .final = '~' }, + [TERM_KEY_F9] = { .tilde = 20, .final = '~' }, + [TERM_KEY_F10] = { .tilde = 21, .final = '~' }, + [TERM_KEY_F11] = { .tilde = 23, .final = '~' }, + [TERM_KEY_F12] = { .tilde = 24, .final = '~' }, + + [TERM_KEY_KP_0] = { .keypad_ss3 = 'p' }, + [TERM_KEY_KP_1] = { .keypad_ss3 = 'q' }, + [TERM_KEY_KP_2] = { .keypad_ss3 = 'r' }, + [TERM_KEY_KP_3] = { .keypad_ss3 = 's' }, + [TERM_KEY_KP_4] = { .keypad_ss3 = 't' }, + [TERM_KEY_KP_5] = { .keypad_ss3 = 'u' }, + [TERM_KEY_KP_6] = { .keypad_ss3 = 'v' }, + [TERM_KEY_KP_7] = { .keypad_ss3 = 'w' }, + [TERM_KEY_KP_8] = { .keypad_ss3 = 'x' }, + [TERM_KEY_KP_9] = { .keypad_ss3 = 'y' }, + [TERM_KEY_KP_DECIMAL] = { .keypad_ss3 = 'n' }, + [TERM_KEY_KP_DIVIDE] = { .keypad_ss3 = 'o' }, + [TERM_KEY_KP_MULTIPLY] = { .keypad_ss3 = 'j' }, + [TERM_KEY_KP_SUBTRACT] = { .keypad_ss3 = 'm' }, + [TERM_KEY_KP_ADD] = { .keypad_ss3 = 'k' }, + [TERM_KEY_KP_ENTER] = { .keypad_ss3 = 'M', .equiv = TERM_KEY_RETURN }, + [TERM_KEY_KP_LEFT] = { .equiv = TERM_KEY_LEFT }, + [TERM_KEY_KP_RIGHT] = { .equiv = TERM_KEY_RIGHT }, + [TERM_KEY_KP_UP] = { .equiv = TERM_KEY_UP }, + [TERM_KEY_KP_DOWN] = { .equiv = TERM_KEY_DOWN }, + [TERM_KEY_KP_PAGE_UP] = { .equiv = TERM_KEY_PAGE_UP }, + [TERM_KEY_KP_PAGE_DOWN] = { .equiv = TERM_KEY_PAGE_DOWN }, + [TERM_KEY_KP_HOME] = { .equiv = TERM_KEY_HOME }, + [TERM_KEY_KP_END] = { .equiv = TERM_KEY_END }, + [TERM_KEY_KP_INSERT] = { .equiv = TERM_KEY_INSERT }, + [TERM_KEY_KP_DELETE] = { .equiv = TERM_KEY_DELETE }, + [TERM_KEY_KP_BEGIN] = { .tilde = 0, .final = 'E' }, +}; + +enum input_mod_key { + MOD_KEY_SHIFT = 0, + MOD_KEY_CTRL, + MOD_KEY_ALT, + MOD_KEY_SUPER, + MOD_KEY_HYPER, + MOD_KEY_META, + MOD_KEY_ISO3, + MOD_KEY_ISO5, + MOD_KEY_COUNT, +}; + +struct mod_desc { + enum input_mod_key group; + enum term_key partner; +}; + +static const struct mod_desc mod_desc[TERM_KEY_COUNT] = { + [TERM_KEY_LSHIFT] = { MOD_KEY_SHIFT, TERM_KEY_RSHIFT }, + [TERM_KEY_RSHIFT] = { MOD_KEY_SHIFT, TERM_KEY_LSHIFT }, + [TERM_KEY_LCTRL] = { MOD_KEY_CTRL, TERM_KEY_RCTRL }, + [TERM_KEY_RCTRL] = { MOD_KEY_CTRL, TERM_KEY_LCTRL }, + [TERM_KEY_LALT] = { MOD_KEY_ALT, TERM_KEY_RALT }, + [TERM_KEY_RALT] = { MOD_KEY_ALT, TERM_KEY_LALT }, + [TERM_KEY_LSUPER] = { MOD_KEY_SUPER, TERM_KEY_RSUPER }, + [TERM_KEY_RSUPER] = { MOD_KEY_SUPER, TERM_KEY_LSUPER }, + [TERM_KEY_LHYPER] = { MOD_KEY_HYPER, TERM_KEY_RHYPER }, + [TERM_KEY_RHYPER] = { MOD_KEY_HYPER, TERM_KEY_LHYPER }, + [TERM_KEY_LMETA] = { MOD_KEY_META, TERM_KEY_RMETA }, + [TERM_KEY_RMETA] = { MOD_KEY_META, TERM_KEY_LMETA }, + [TERM_KEY_ISO_LEVEL3_SHIFT] = { MOD_KEY_ISO3, TERM_KEY_NONE }, + [TERM_KEY_ISO_LEVEL5_SHIFT] = { MOD_KEY_ISO5, TERM_KEY_NONE }, +}; + +static const unsigned mod_group_input_mod[MOD_KEY_COUNT] = { + TERM_MOD_SHIFT, TERM_MOD_CTRL, TERM_MOD_ALT, + TERM_MOD_SUPER, TERM_MOD_HYPER, TERM_MOD_META, + 0, 0, +}; + +static inline bool key_is_modifier(enum term_key k) +{ + return (unsigned)k < TERM_KEY_COUNT && + (mod_desc[k].partner != TERM_KEY_NONE || + mod_desc[k].group == MOD_KEY_ISO3 || + mod_desc[k].group == MOD_KEY_ISO5); +} + +static inline bool key_is_lock(enum term_key k) +{ + return k == TERM_KEY_CAPS_LOCK || k == TERM_KEY_SCROLL_LOCK || + k == TERM_KEY_NUM_LOCK; +} + +static unsigned effective_mods(const struct term_keyboard_state *s, + const struct term_key_event *ev) +{ + unsigned mods = ev->mods; + if (!key_is_modifier(ev->key)) return mods; + + const struct mod_desc *d = &mod_desc[ev->key]; + if (d->group >= MOD_KEY_ISO3) return mods; + + unsigned bit = mod_group_input_mod[d->group]; + bool other_held = term_keyboard_key_down(s, d->partner); + if (ev->action != TERM_KEY_RELEASE || other_held) mods |= bit; + else mods &= ~bit; + return mods; +} + +static unsigned lock_mod_bit(enum term_key key) +{ + if (key == TERM_KEY_CAPS_LOCK) return TERM_MOD_CAPS; + if (key == TERM_KEY_NUM_LOCK) return TERM_MOD_NUM; + return 0; +} + +static unsigned toggle_locks(enum term_key key, enum term_key_action action, + unsigned mods) +{ + unsigned bit = lock_mod_bit(key); + if (bit && action == TERM_KEY_PRESS) mods ^= bit; + return mods; +} + +/* Convert logical input modifiers to terminal protocol bits. */ +static unsigned protocol_modifiers(unsigned mods) +{ + return ((mods & TERM_MOD_SHIFT) ? 1u : 0u) | + ((mods & TERM_MOD_ALT) ? 2u : 0u) | + ((mods & TERM_MOD_CTRL) ? 4u : 0u) | + ((mods & TERM_MOD_SUPER) ? 8u : 0u) | + ((mods & TERM_MOD_HYPER) ? 16u : 0u) | + ((mods & TERM_MOD_META) ? 32u : 0u) | + ((mods & TERM_MOD_CAPS) ? 64u : 0u) | + ((mods & TERM_MOD_NUM) ? 128u : 0u); +} + +static void format_kitty(struct seqbuf *sb, uint32_t code, char final, + unsigned flags, bool functional, uint32_t shifted, + uint32_t base, unsigned mods, unsigned event, + bool report_text, uint32_t text) +{ + char tmp[64]; + struct seqbuf body; + sb_init(&body, tmp, sizeof(tmp)); + + if (final != 'u' && final != '~' && code <= 1 && !shifted && !base && + !mods && !(flags & TERM_KEYBOARD_EVENT_TYPES) && !report_text) { + + sb_ch(&body, '\033'); + sb_ch(&body, '['); + sb_ch(&body, final); + } else { + sb_ch(&body, '\033'); + sb_ch(&body, '['); + sb_u32(&body, code); + if ((flags & TERM_KEYBOARD_ALTERNATE_KEYS) && !functional) { + if (shifted) { + sb_ch(&body, ':'); + sb_u32(&body, shifted); + } + if (base) { + sb_ch(&body, ':'); + if (!shifted) sb_ch(&body, ':'); + sb_u32(&body, base); + } + } + if (mods || (flags & TERM_KEYBOARD_EVENT_TYPES) || report_text) { + sb_ch(&body, ';'); + sb_u32(&body, mods + 1); + } + if ((flags & TERM_KEYBOARD_EVENT_TYPES) && event != 1) { + sb_ch(&body, ':'); + sb_u32(&body, event); + } + if (report_text) { + sb_ch(&body, ';'); + sb_u32(&body, text); + } + sb_ch(&body, final); + } + + if (body.overflow) { + sb->overflow = true; + return; + } + sb_str(sb, tmp, sb_len(&body, tmp)); +} + +static bool encode_kitty(const struct term_keyboard_state *s, + const struct term_key_event *ev, + const struct term_modes *m, + struct seqbuf *sb) +{ + unsigned flags = m->keyboard; + unsigned event = ev->action == TERM_KEY_PRESS ? 1u : + ev->action == TERM_KEY_REPEAT ? 2u : 3u; + if (!flags) return ev->action == TERM_KEY_RELEASE; + + const struct kitty_desc *d = &kitty_desc[ev->key]; + bool functional = d->code != 0; + char final = functional ? d->final : 'u'; + bool all = (flags & TERM_KEYBOARD_ALL_KEYS) != 0; + + if ((key_is_modifier(ev->key) || key_is_lock(ev->key)) && !all) + return true; + + unsigned mods = effective_mods(s, ev); + mods = toggle_locks(ev->key, ev->action, mods); + + bool plain_control = ev->key == TERM_KEY_RETURN || ev->key == TERM_KEY_TAB || + ev->key == TERM_KEY_BACKSPACE; + if (ev->action == TERM_KEY_RELEASE && + (!(flags & TERM_KEYBOARD_EVENT_TYPES) || (!all && plain_control))) + return true; + + const unsigned command_mods = TERM_MOD_ALT | TERM_MOD_CTRL | + TERM_MOD_SUPER | TERM_MOD_HYPER | + TERM_MOD_META; + uint32_t text = ev->text; + bool has_text = text >= 32 && !(text >= 127 && text < 160) && + !(mods & command_mods); + if (!all && has_text) return ev->action == TERM_KEY_RELEASE; + if (!all && plain_control && !(protocol_modifiers(mods) & 63)) + return ev->action == TERM_KEY_RELEASE; + if (!(flags & (TERM_KEYBOARD_DISAMBIGUATE | TERM_KEYBOARD_EVENT_TYPES | TERM_KEYBOARD_ALL_KEYS))) + return ev->action == TERM_KEY_RELEASE; + + uint32_t code = d->code, shifted = 0, base = 0; + if (!functional) { + code = ev->codepoint; + if (flags & TERM_KEYBOARD_ALTERNATE_KEYS) { + shifted = (mods & TERM_MOD_SHIFT) ? ev->shifted : 0; + base = ev->base; + if (shifted == code) shifted = 0; + if (base == code) base = 0; + } + } + if (!code) return true; + + bool report_text = all && (flags & TERM_KEYBOARD_ASSOCIATED_TEXT) && has_text && + ev->action != TERM_KEY_RELEASE; + format_kitty(sb, code, final, flags, functional, shifted, base, protocol_modifiers(mods), + event, report_text, text); + return true; +} + +static uint32_t control_base_char(const struct term_key_event *ev) +{ + uint32_t ch = ((ev->mods & TERM_MOD_SHIFT) && ev->shifted) + ? ev->shifted : ev->codepoint; + return ch < 0x80 ? ch : 0; +} + +static bool encode_legacy(const struct term_key_event *ev, + const struct term_modes *m, + struct seqbuf *sb) +{ + if (m->app_keypad && (unsigned)ev->keypad < TERM_KEY_COUNT && + legacy_desc[ev->keypad].keypad_ss3) { + sb_ch(sb, '\033'); + sb_ch(sb, 'O'); + sb_ch(sb, legacy_desc[ev->keypad].keypad_ss3); + return true; + } + + uint32_t ch = control_base_char(ev); + if ((ev->mods & TERM_MOD_CTRL) && ch) { + if ((ch >= 'a' && ch <= 'z') || (ch >= '@' && ch <= '_')) { + sb_ch(sb, (char)(ch & 0x1f)); + return true; + } + if (ch == ' ') { sb_ch(sb, 0); return true; } + if (ch == '-') { sb_ch(sb, 0x1f); return true; } + } + + unsigned csi = protocol_modifiers(ev->mods) & 63; + unsigned p = csi + 1; + bool shift = (ev->mods & TERM_MOD_SHIFT) != 0; + + enum term_key k = ev->key; + if ((unsigned)k < TERM_KEY_COUNT && legacy_desc[k].equiv != TERM_KEY_NONE) + k = legacy_desc[k].equiv; + + if ((unsigned)k < TERM_KEY_COUNT) { + const struct legacy_desc *d = &legacy_desc[k]; + if (d->final) { + if (d->tilde) { + sb_ch(sb, '\033'); + sb_ch(sb, '['); + sb_u32(sb, d->tilde); + if (csi) { + sb_ch(sb, ';'); + sb_u32(sb, p); + } + sb_ch(sb, '~'); + return true; + } + if (csi) { + sb_ch(sb, '\033'); + sb_ch(sb, '['); + sb_ch(sb, '1'); + sb_ch(sb, ';'); + sb_u32(sb, p); + sb_ch(sb, d->final); + return true; + } + sb_ch(sb, '\033'); + sb_ch(sb, d->app_cursor && !m->app_cursor ? '[' : 'O'); + sb_ch(sb, d->final); + return true; + } + } + + switch (k) { + case TERM_KEY_RETURN: sb_ch(sb, '\r'); return true; + case TERM_KEY_BACKSPACE: + sb_ch(sb, m->backarrow_bs ? '\x08' : '\x7f'); + return true; + case TERM_KEY_TAB: + if (shift) { sb_str(sb, "\033[Z", 3); } + else { sb_ch(sb, '\t'); } + return true; + case TERM_KEY_ESCAPE: sb_ch(sb, '\033'); return true; + default: break; + } + + if (ev->text && !ev->text_deferred) { + char tmp[8]; + size_t n = (size_t)utf8_encode(ev->text, tmp); + sb_str(sb, tmp, n); + return true; + } + return false; +} + +void term_keyboard_update(struct term_keyboard_state *s, const struct term_key_event *ev) +{ + if (!s || !ev || ev->key <= TERM_KEY_NONE || ev->key >= TERM_KEY_COUNT) + return; + if (ev->action == TERM_KEY_RELEASE) + s->down[ev->key >> 6] &= ~(1ull << (ev->key & 63)); + else if (ev->action == TERM_KEY_PRESS || ev->action == TERM_KEY_REPEAT) + s->down[ev->key >> 6] |= 1ull << (ev->key & 63); +} + +bool term_keyboard_key_down(const struct term_keyboard_state *s, enum term_key key) +{ + return s && key > TERM_KEY_NONE && (unsigned)key < TERM_KEY_COUNT && + ((s->down[key >> 6] >> (key & 63)) & 1); +} + +struct term_input_result term_input_encode_key(const struct term_keyboard_state *s, + const struct term_key_event *ev, + const struct term_modes *m, + char *buf, size_t buf_sz) +{ + struct term_input_result out = {0}; + if (!s || !ev || !m || !buf || + (ev->action != TERM_KEY_PRESS && ev->action != TERM_KEY_REPEAT && + ev->action != TERM_KEY_RELEASE) || + (unsigned)ev->key >= TERM_KEY_COUNT) + return out; + + struct seqbuf sb; + sb_init(&sb, buf, buf_sz); + + bool handled = encode_kitty(s, ev, m, &sb); + if (!handled) { + if (ev->action == TERM_KEY_RELEASE) + return out; + + bool alt = (ev->mods & TERM_MOD_ALT) != 0; + struct seqbuf body; + char tmp[64]; + if (alt) { + sb_init(&body, tmp, sizeof(tmp)); + } else { + body = sb; + } + + handled = encode_legacy(ev, m, &body); + if (handled) { + if (body.overflow) { + sb.overflow = true; + } else { + size_t n = sb_len(&body, alt ? tmp : buf); + if (alt) { + bool already_esc = n && tmp[0] == '\033'; + size_t prefix = !already_esc; + if (n + prefix <= buf_sz) { + if (prefix) buf[0] = '\033'; + memcpy(buf + prefix, tmp, n); + n += prefix; + } else { + sb.overflow = true; + n = 0; + } + } + out.len = n; + } + } + } else { + out.len = sb_len(&sb, buf); + } + + if (!handled) + return out; + if (sb.overflow) { + out.status = TERM_INPUT_BUFFER_FULL; + out.len = 0; + } else { + out.status = out.len ? TERM_INPUT_BYTES : TERM_INPUT_CONSUMED; + } + return out; +} + +struct term_input_result term_input_encode_mouse(const struct term_mouse_event *ev, + const struct term_modes *m, + char *buf, size_t buf_sz) +{ + struct term_input_result out = {0}; + if (!ev || !m || !buf) return out; + + if (!m->mouse || (ev->mods & TERM_MOD_SHIFT)) return out; + + out.status = TERM_INPUT_CONSUMED; + if (ev->kind == TERM_MOUSE_LEAVE) return out; + struct term_vector2i cell = ev->pos; + if (cell.x < 0 || cell.y < 0 || cell.x >= m->grid.x || cell.y >= m->grid.y) + return out; + if (m->mouse == TERM_MOUSE_X10 && (ev->kind == TERM_MOUSE_RELEASE || ev->kind == TERM_MOUSE_MOTION)) { + return out; + } + if (ev->kind == TERM_MOUSE_MOTION && (m->mouse == TERM_MOUSE_NORMAL || + (m->mouse == TERM_MOUSE_BUTTON && ev->button == 3))) { + return out; + } + int button; + if (ev->button >= TERM_MOUSE_BUTTON_PRIMARY && ev->button <= TERM_MOUSE_BUTTON_NONE) + button = ev->button; + else if (ev->button >= TERM_MOUSE_BUTTON_EXTRA_1 && ev->button <= TERM_MOUSE_BUTTON_EXTRA_5) + button = 8 + ev->button - TERM_MOUSE_BUTTON_EXTRA_1; + else if (ev->button >= TERM_MOUSE_WHEEL_UP && ev->button <= TERM_MOUSE_WHEEL_RIGHT) + button = 64 + ev->button - TERM_MOUSE_WHEEL_UP; + else { + return out; + } + + int mods = 0; + if (ev->mods & TERM_MOD_ALT) mods |= 8; + if (ev->mods & TERM_MOD_CTRL) mods |= 16; + + int code = button | mods | (ev->kind == TERM_MOUSE_MOTION ? 32 : 0); + int legacy_code = ev->kind == TERM_MOUSE_RELEASE ? 3 | mods : code; + struct term_vector2i report = m->mouse_encoding == TERM_MOUSE_ENC_SGR_PIXELS + ? ev->pixel : cell; + if (report.x < 0 || report.y < 0) + return out; + uint32_t report_x = (uint32_t)report.x + 1; + uint32_t report_y = (uint32_t)report.y + 1; + + struct seqbuf sb; + sb_init(&sb, buf, buf_sz); + + if (m->mouse_encoding == TERM_MOUSE_ENC_SGR || + m->mouse_encoding == TERM_MOUSE_ENC_SGR_PIXELS) { + sb_ch(&sb, '\033'); + sb_ch(&sb, '['); + sb_ch(&sb, '<'); + sb_u32(&sb, (uint32_t)code); + sb_ch(&sb, ';'); + sb_u32(&sb, (uint32_t)report_x); + sb_ch(&sb, ';'); + sb_u32(&sb, (uint32_t)report_y); + sb_ch(&sb, ev->kind == TERM_MOUSE_RELEASE ? 'm' : 'M'); + } else if (m->mouse_encoding == TERM_MOUSE_ENC_URXVT) { + sb_ch(&sb, '\033'); + sb_ch(&sb, '['); + sb_u32(&sb, (uint32_t)(legacy_code + 32)); + sb_ch(&sb, ';'); + sb_u32(&sb, (uint32_t)(cell.x + 1)); + sb_ch(&sb, ';'); + sb_u32(&sb, (uint32_t)(cell.y + 1)); + sb_ch(&sb, 'M'); + } else if (m->mouse_encoding == TERM_MOUSE_ENC_UTF8) { + const unsigned values[3] = { + (unsigned)legacy_code + 32, (unsigned)cell.x + 33, + (unsigned)cell.y + 33, + }; + sb_str(&sb, "\033[M", 3); + for (int i = 0; i < 3; i++) { + char tmp[8]; + size_t n = (size_t)utf8_encode(values[i], tmp); + sb_str(&sb, tmp, n); + } + } else { + if (legacy_code > 223 || cell.x > 222 || cell.y > 222) + return out; + sb_ch(&sb, '\033'); + sb_ch(&sb, '['); + sb_ch(&sb, 'M'); + sb_ch(&sb, (char)(legacy_code + 32)); + sb_ch(&sb, (char)(cell.x + 33)); + sb_ch(&sb, (char)(cell.y + 33)); + } + + if (sb.overflow) { + out.status = TERM_INPUT_BUFFER_FULL; + out.len = 0; + return out; + } + out.len = sb_len(&sb, buf); + out.status = TERM_INPUT_BYTES; + return out; +} + +/* Bidi controls can make pasted text look different from what runs. */ +static bool is_direction_format(uint32_t cp) +{ + return cp == 0x061c || + (cp >= 0x200e && cp <= 0x200f) || + (cp >= 0x202a && cp <= 0x202e) || + (cp >= 0x2066 && cp <= 0x2069) || + cp == 0xfeff; +} + +struct term_input_result term_input_encode_paste(const struct term_modes *m, + const char *text, size_t len, + char *buf, size_t capacity) +{ + struct term_input_result out = {0}; + if (!m || !buf || (!text && len)) return out; + + struct seqbuf sb; + sb_init(&sb, buf, capacity); + if (m->bracketed_paste) sb_bytes(&sb, "\033[200~", 6); + size_t body = sb_len(&sb, buf); + for (size_t i = 0; i < len;) { + size_t start = i; + while (i < len && (unsigned char)text[i] >= 0x20 && (unsigned char)text[i] < 0x7f) + i++; + if (i > start) { + sb_bytes(&sb, text + start, i - start); + continue; + } + uint32_t cp; + int n = utf8_decode_one(text + i, len - i, &cp); + if (!n) { i++; continue; } + if (cp == '\r' || cp == '\n') { + sb_ch(&sb, '\r'); + if (cp == '\r' && i + 1 < len && text[i + 1] == '\n') i++; + } else if (cp == '\t') { + sb_ch(&sb, '\t'); + } else if (cp >= 0xa0 && !is_direction_format(cp)) { + sb_bytes(&sb, text + i, (size_t)n); + } + i += (size_t)n; + } + + out.status = TERM_INPUT_CONSUMED; + if (!sb.overflow && sb_len(&sb, buf) == body) return out; + if (m->bracketed_paste) sb_bytes(&sb, "\033[201~", 6); + if (sb.overflow) { + out.status = TERM_INPUT_BUFFER_FULL; + return out; + } + out.len = sb_len(&sb, buf); + out.status = TERM_INPUT_BYTES; + return out; +} diff --git a/src/internal.h b/src/internal.h new file mode 100644 index 0000000..dd4649c --- /dev/null +++ b/src/internal.h @@ -0,0 +1,411 @@ +#ifndef TERM_INTERNAL_H +#define TERM_INTERNAL_H + +#include "term.h" +#include "cell.h" +#include "csi.h" +#include "gstore.h" +#include "hist.h" +#include "hist_view.h" +#include "palette.h" +#include "pen.h" +#include "text_run.h" +#include "compat.h" +#include <stdatomic.h> +#include <stdlib.h> +#include <string.h> + +#define HASH64_BASIS UINT64_C(0x9E3779B97F4A7C15) +#define KNUTH_HASH_MULT UINT32_C(2654435761) + +static inline uint64_t hash64_step(uint64_t hash, uint32_t value) +{ + hash ^= value; + hash *= HASH64_BASIS; + return hash ^ (hash >> 29); +} + +static inline uint64_t hash64_finish(uint64_t hash) +{ + hash ^= hash >> 32; + hash *= UINT64_C(0xD6E8FEB86659FD93); + hash ^= hash >> 32; + return hash; +} + +#define KEYBOARD_STACK_SIZE 32 + +struct keyboard_mode { + unsigned flags, saved[KEYBOARD_STACK_SIZE], depth; +}; + +/* What DECSC saves, as in xterm: the cursor, the pen, origin mode, the + * pending wrap, and the character sets. */ +struct term_saved_cursor { + int x, y; + uint32_t fg, bg, ul; + char charsets[2]; + uint8_t underline; + uint8_t charset_active; + bool bold : 1; + bool faint : 1; + bool italic : 1; + bool strike : 1; + bool overline : 1; + bool inverse : 1; + bool origin : 1; + bool pending_wrap : 1; +}; + +/* + * A row stores cells, text, or default-pen ASCII. A text row uses the history + * format. An ASCII row stores only its pointer and length. A screen read or + * edit sets its text fields first. Other changes settle text into cells. + */ +enum { ROW_CELLS, ROW_TEXT, ROW_ASCII }; +enum { RF_WRAPPED = 1, RF_WIDE = 4, RF_CLUSTER = 8, RF_TAIL_PADDED = 16 }; + +struct srow { + struct cell *cells; /* stride >= cols, from a per-screen slab */ + uint8_t *ascii; + struct text_row text; /* TEXT: the content; len == text.cells, or cols with a wrap pad */ + uint64_t uid; /* new id when the row scrolls in or clears */ + uint32_t version; /* bumped on every change */ + uint16_t len; + uint8_t kind, flags; +}; + +struct screen { + struct srow *slot; + struct cell *cells_slab; + uint8_t *text_slab; + size_t cells_cap, text_cap; + /* Without history: the line of row 0, counted as rows leave the top. */ + uint64_t top_id; + int rows, cols, top; + int stride; + /* Capacity of the buffers above. A resize reuses them. */ + int slot_cap; + uint32_t top_seg; +}; + +struct term_sel_point { + uint64_t line_id; + uint32_t cell_off; +}; + +/* A buffer that only grows, so work that repeats does not allocate. */ +struct grow_buf { + void *data; + size_t cap; +}; + +/* What a resize needs besides the screens. It keeps its buffers, so a + * resize allocates nothing once the terminal has seen its sizes. */ +struct resize_scratch { + struct grow_buf row, rows, cells, line, pull, rest, lens; +}; + + +struct term { + _Atomic uint64_t seq; + _Atomic uint64_t writer_tid; + uint32_t write_depth; + int cols; + int rows; + int cursor_x; + int cursor_y; + int scroll_top, scroll_bottom; + + uint32_t cur_fg; + uint32_t cur_bg; + uint32_t cur_ul; + uint32_t last_char; + + uint32_t utf8_state; + uint32_t utf8_codep; + struct width_cache text_width; + + enum { ST_GROUND, ST_ESC, ST_CSI, ST_OSC, ST_DCS, ST_STR_ESC, ST_CHARSET, + ST_DCS_ESC, ST_IGNORE, ST_IGNORE_ESC, ST_DOCS, ST_DECALN } state; + struct csi_parser csi_parser; + + uint16_t cur_link; + char charsets[2]; + uint8_t underline; + uint8_t decsace; + char charset_designation; + bool graphics; + + bool bold : 1; + bool faint : 1; + bool italic : 1; + bool strike : 1; + bool overline : 1; + bool inverse : 1; + bool pending_wrap : 1; + bool autowrap : 1; + bool insert_mode : 1; + bool origin_mode : 1; + bool reverse_video : 1; + bool synchronized_output : 1; + bool alternate : 1; + bool application_cursor : 1; + bool application_keypad : 1; + bool lr_margin_mode : 1; + bool backarrow_sends_bs : 1; + bool focus_events : 1; + bool cursor_visible : 1; + bool cursor_blink : 1; + bool default_cursor_blink : 1; + bool selection : 1; + bool bracketed_paste : 1; + bool lnm : 1; + bool transport_newline : 1; + bool str_truncated : 1; + bool has_prompt : 1; + + struct screen scr; + struct screen alt_scr; + struct screen spare; /* the buffers a resize builds into */ + struct resize_scratch resize; + struct grow_buf ascii_ends; + struct grow_buf ascii_tail; + struct term_saved_cursor normal_saved_cursor; + struct term_saved_cursor normal_saved_decsc; + int normal_cols, normal_rows; + bool alt_allocated; + bool normal_cursor_visible; + uint8_t normal_cursor_style; + bool normal_cursor_blink; + + struct hist hist; + struct hist_view view; + uint64_t next_row_uid; + uint64_t full_gen; + int prompt_y; + int left_margin, right_margin; + struct term_sel_point sel_start, sel_end; + struct term_sel_point sel_anchor_first, sel_anchor_last; + enum term_select_unit select_unit; + + int mouse_mode, mouse_encoding; + enum term_cursor_style cursor_style; + enum term_cursor_style default_cursor_style; + int charset_active; + int osc_len; + int dcs_len; + int sb_capacity; + + struct keyboard_mode keyboard[2]; + /* The DECSC slot. Cold: it stays out of the hot fields above. */ + struct term_saved_cursor saved; + + uint8_t *tabs; + struct cell *view_row_scratch; + int tabs_cap; + int view_scratch_cap; + + char osc_buf[256]; + char tcap_name[64]; + + char *osc_heap; + size_t osc_heap_cap; + char *dcs_buf; + size_t dcs_cap; + void *links; + char *cwd; + char *pointer_shape; + + struct term_callbacks cb; + struct palette palette; + struct gstore *gstore; + /* Unique for the life of the process. A snapshot compares it, not the + * address, because a new terminal can reuse a freed one's memory. */ + uint64_t serial; +}; + +static inline void reset_sgr(struct term *t) +{ + t->cur_fg = TERM_COLOR_FG; + t->cur_bg = TERM_COLOR_BG; + t->bold = false; + t->faint = false; + t->italic = false; + t->underline = 0; + t->strike = false; + t->overline = false; + t->inverse = false; + t->cur_ul = 0; +} + +/* The DECSC slot before any save: home, the default pen, and ASCII. */ +static inline struct term_saved_cursor term_saved_cursor_initial(void) +{ + return (struct term_saved_cursor){ + .fg = TERM_COLOR_FG, .bg = TERM_COLOR_BG, .charsets = { 'B', 'B' }, + }; +} + +static inline struct term_saved_cursor term_cursor_save(const struct term *t) +{ + return (struct term_saved_cursor){ + .x = t->cursor_x, .y = t->cursor_y, + .fg = t->cur_fg, .bg = t->cur_bg, .ul = t->cur_ul, + .charsets = { t->charsets[0], t->charsets[1] }, + .underline = t->underline, + .charset_active = (uint8_t)t->charset_active, + .bold = t->bold, .faint = t->faint, .italic = t->italic, + .strike = t->strike, .overline = t->overline, .inverse = t->inverse, + .origin = t->origin_mode, .pending_wrap = t->pending_wrap, + }; +} + +/* Take the pen, origin mode, and character sets from s. The caller places + * the cursor. */ +static inline void term_cursor_apply(struct term *t, const struct term_saved_cursor *s) +{ + t->cur_fg = s->fg; + t->cur_bg = s->bg; + t->cur_ul = s->ul; + t->underline = s->underline; + t->bold = s->bold; + t->faint = s->faint; + t->italic = s->italic; + t->strike = s->strike; + t->overline = s->overline; + t->inverse = s->inverse; + t->origin_mode = s->origin; + t->charsets[0] = s->charsets[0]; + t->charsets[1] = s->charsets[1]; + t->charset_active = s->charset_active; + t->graphics = t->charsets[t->charset_active] == '0'; +} + +static inline void send_response(struct term *t, const char *str) +{ + if (t && t->cb.reply) + t->cb.reply(t->cb.ud, str, strlen(str)); +} +static inline int term_hex_nib(char c) +{ + if (c >= '0' && c <= '9') return c - '0'; + if (c >= 'a' && c <= 'f') return c - 'a' + 10; + if (c >= 'A' && c <= 'F') return c - 'A' + 10; + return -1; +} + +static inline bool term_lnm(const struct term *t) +{ + return t->lnm || t->transport_newline; +} + +static inline bool mode_is_mouse(int m) +{ + return m == 9 || (m >= 1000 && m <= 1003); +} + +static inline bool mode_is_mouse_enc(int m) +{ + return m == 1005 || m == 1006 || m == 1015 || m == 1016; +} + +static const uint16_t term_ul_flags[] = { + 0, CELL_UNDERLINE, CELL_DOUBLE_UNDERLINE, + CELL_CURLY_UNDERLINE, CELL_DOTTED_UNDERLINE, + CELL_DASHED_UNDERLINE, +}; + +static inline uint16_t term_current_flags(const struct term *t) +{ + uint16_t f = 0; + if (t->bold) f |= CELL_BOLD; + if (t->faint) f |= CELL_FAINT; + if (t->italic) f |= CELL_ITALIC; + if (t->underline < sizeof(term_ul_flags) / sizeof(term_ul_flags[0])) + f |= term_ul_flags[t->underline]; + if (t->strike) f |= CELL_STRIKE; + if (t->overline) f |= CELL_OVERLINE; + if (t->cur_link) f |= CELL_LINK; + return f; +} + +static inline uint32_t term_flags_pad(const struct term *t) +{ + return term_current_flags(t) | ((uint32_t)t->cur_link << 16); +} + +static inline void term_tabs_clear(struct term *t) +{ + memset(t->tabs, 0, (size_t)t->tabs_cap); +} + +static inline void term_tabs_default(struct term *t) +{ + for (int c = 8; c < t->tabs_cap; c += 8) t->tabs[c] = 1; +} + +static inline bool term_str_was_truncated(struct term *t) +{ + bool truncated = t->str_truncated; + t->str_truncated = false; + return truncated; +} + +/* Copy text into buf with the rules of term_selection_text(). NULL text + * copies as empty. Return the full length. */ +static inline size_t term_copy_out(const char *text, char *buf, size_t cap) +{ + if (!text) + text = ""; + size_t n = strlen(text); + if (buf && cap) { + size_t k = n < cap ? n : cap - 1; + if (k) memcpy(buf, text, k); + buf[k] = '\0'; + } + return n; +} + +/* Every thread has its own anchor, so its address names the thread. */ +extern _Thread_local char term_thread_anchor; + +static inline uint64_t term_thread_id(void) +{ + return (uint64_t)(uintptr_t)&term_thread_anchor; +} + +static inline void term_write_begin(struct term *t) +{ + if (!t) return; + uint64_t self = term_thread_id(); + if (atomic_load_explicit(&t->writer_tid, memory_order_relaxed) == self) { + t->write_depth++; + return; + } + uint64_t s; + for (;;) { + s = atomic_load_explicit(&t->seq, memory_order_relaxed); + if (s & 1) { + term_cpu_relax(); + continue; + } + if (atomic_compare_exchange_weak_explicit(&t->seq, &s, s + 1, + memory_order_acquire, + memory_order_relaxed)) + break; + } + atomic_store_explicit(&t->writer_tid, self, memory_order_relaxed); + t->write_depth = 1; +} + +static inline void term_write_end(struct term *t) +{ + if (!t) return; + if (--t->write_depth == 0) { + atomic_store_explicit(&t->writer_tid, 0, memory_order_relaxed); + atomic_fetch_add_explicit(&t->seq, 1, memory_order_release); + } +} + +#endif diff --git a/src/isa.c b/src/isa.c new file mode 100644 index 0000000..1fe1517 --- /dev/null +++ b/src/isa.c @@ -0,0 +1,78 @@ +/* + * The instruction set level. See simd.h. The level is picked once, before + * the first terminal exists, so every call in a process uses the same + * kernels. + */ +#include "term.h" +#include "simd.h" + +#include <stdatomic.h> +#include <stdlib.h> +#include <string.h> + +int term_isa_level; +static atomic_int isa_state; + +static const char *const level_names[] = { "base", "avx2", "avx512" }; + +#if defined(TERM_ISA_RUNTIME) +/* The AVX-512 kernels also need VBMI and VBMI2. Every CPU with VBMI except + * Cannon Lake has VBMI2. */ +static int cpu_level(void) +{ + __builtin_cpu_init(); + if (__builtin_cpu_supports("x86-64-v4") && __builtin_cpu_supports("avx512vbmi") && + __builtin_cpu_supports("avx512vbmi2")) + return TERM_ISA_AVX512; + if (__builtin_cpu_supports("x86-64-v3")) + return TERM_ISA_AVX2; + return TERM_ISA_BASE; +} +#else +static int cpu_level(void) +{ + return TERM_ISA_FLOOR; +} +#endif + +static int pick_level(void) +{ + int level = cpu_level(); + /* TERM_HEADLESS_ISA can only lower the level, never below the floor the + * compiler flags fix. */ + const char *want = getenv("TERM_HEADLESS_ISA"); + for (int i = TERM_ISA_FLOOR; want && i < level; i++) + if (!strcmp(want, level_names[i])) + level = i; + return level; +} + +/* The first caller picks the level. A second caller waits until it is set. */ +void term_isa_init(void) +{ + if (atomic_load_explicit(&isa_state, memory_order_acquire) == 2) + return; + int expected = 0; + if (atomic_compare_exchange_strong(&isa_state, &expected, 1)) { + term_isa_level = pick_level(); + atomic_store_explicit(&isa_state, 2, memory_order_release); + return; + } + while (atomic_load_explicit(&isa_state, memory_order_acquire) != 2) + term_cpu_relax(); +} + +const char *term_isa(void) +{ + term_isa_init(); +#if defined(__aarch64__) + return "neon"; +#else + int level = TERM_ISA_FLOOR; + if (term_use_avx2()) + level = TERM_ISA_AVX2; + if (term_use_avx512()) + level = TERM_ISA_AVX512; + return level_names[level]; +#endif +} diff --git a/src/kernels.h b/src/kernels.h new file mode 100644 index 0000000..48088b5 --- /dev/null +++ b/src/kernels.h @@ -0,0 +1,790 @@ +#ifndef WSAP_TERM_KERNELS_H +#define WSAP_TERM_KERNELS_H + +#include "cell.h" +#include "compat.h" +#include "simd.h" + +#include <stdbool.h> +#include <stddef.h> +#include <stdint.h> +#include <string.h> + +#if defined(TERM_KERNEL_AVX512) +TERM_TARGET_AVX512 +static inline size_t scan_printable_span_avx512(const char *buf, size_t len, unsigned char hi) +{ + size_t i = 0; + __m512i v20_512 = _mm512_set1_epi8(0x20); + __m512i vlim_512 = _mm512_set1_epi8((char)(hi - 0x20)); + for (; i + 64 <= len; i += 64) { + __m512i v = _mm512_loadu_si512((const void *)(buf + i)); + __mmask64 mask = _mm512_cmpge_epu8_mask(_mm512_sub_epi8(v, v20_512), vlim_512); + if (mask != 0) + return i + term_ctz64(mask); + } + if (i < len) { + __mmask64 load_mask = ((uint64_t)1 << (len - i)) - 1; + __m512i v = _mm512_maskz_loadu_epi8(load_mask, buf + i); + __mmask64 mask = _mm512_mask_cmpge_epu8_mask(load_mask, _mm512_sub_epi8(v, v20_512), vlim_512); + if (mask != 0) + return i + term_ctz64(mask); + } + return len; +} +#endif + +#if defined(TERM_KERNEL_AVX2) +TERM_TARGET_AVX2 +static inline size_t scan_printable_span_avx2(const char *buf, size_t len, unsigned char hi) +{ + size_t i = 0; + const __m256i v_offset = _mm256_set1_epi8((char)(0x20 + 0x80)); + const __m256i v_limit = _mm256_set1_epi8((char)(hi - 0xa1)); + for (; i + 64 <= len; i += 64) { + __m256i v0 = _mm256_loadu_si256((const __m256i *)(buf + i)); + __m256i v1 = _mm256_loadu_si256((const __m256i *)(buf + i + 32)); + __m256i sub0 = _mm256_sub_epi8(v0, v_offset); + __m256i sub1 = _mm256_sub_epi8(v1, v_offset); + int m0 = _mm256_movemask_epi8(_mm256_cmpgt_epi8(sub0, v_limit)); + int m1 = _mm256_movemask_epi8(_mm256_cmpgt_epi8(sub1, v_limit)); + if (TERM_EXPECT((m0 | m1) != 0, 0)) { + if (m0 != 0) + return i + term_ctz32(m0); + return i + 32 + term_ctz32(m1); + } + } + if (i + 32 <= len) { + __m256i v = _mm256_loadu_si256((const __m256i *)(buf + i)); + __m256i sub = _mm256_sub_epi8(v, v_offset); + int mask = _mm256_movemask_epi8(_mm256_cmpgt_epi8(sub, v_limit)); + if (mask != 0) + return i + term_ctz32(mask); + i += 32; + } + if (i + 16 <= len) { + __m128i v = _mm_loadu_si128((const __m128i *)(buf + i)); + __m128i sub = _mm_sub_epi8(v, _mm_set1_epi8((char)(0x20 + 0x80))); + int mask = _mm_movemask_epi8(_mm_cmpgt_epi8(sub, _mm_set1_epi8((char)(hi - 0xa1)))) & 0xFFFF; + if (mask != 0) + return i + term_ctz32(mask); + i += 16; + } + for (; i < len; i++) { + unsigned char c = (unsigned char)buf[i]; + if (c < 0x20 || c >= hi) + return i; + } + return len; +} +#endif + +static inline size_t scan_printable_span(const char *buf, size_t len, + unsigned char hi) +{ + /* A short span costs less inline than as a call. */ +#if defined(TERM_KERNEL_AVX512) + if (len >= 64 && term_use_avx512()) + return scan_printable_span_avx512(buf, len, hi); +#endif +#if defined(TERM_KERNEL_AVX2) + if (len >= 64 && term_use_avx2()) + return scan_printable_span_avx2(buf, len, hi); +#endif + size_t i = 0; +#if defined(__aarch64__) + uint8x16_t v20 = vdupq_n_u8(0x20); + uint8x16_t vlim = vdupq_n_u8((uint8_t)(hi - 0x20)); + for (; i + 64 <= len; i += 64) { + uint8x16_t v0 = vld1q_u8((const uint8_t *)buf + i); + uint8x16_t v1 = vld1q_u8((const uint8_t *)buf + i + 16); + uint8x16_t v2 = vld1q_u8((const uint8_t *)buf + i + 32); + uint8x16_t v3 = vld1q_u8((const uint8_t *)buf + i + 48); + + uint8x16_t m0 = vcgeq_u8(vsubq_u8(v0, v20), vlim); + uint8x16_t m1 = vcgeq_u8(vsubq_u8(v1, v20), vlim); + uint8x16_t m2 = vcgeq_u8(vsubq_u8(v2, v20), vlim); + uint8x16_t m3 = vcgeq_u8(vsubq_u8(v3, v20), vlim); + + uint8x16_t mor = vorrq_u8(vorrq_u8(m0, m1), vorrq_u8(m2, m3)); + uint64x2_t mor64 = vreinterpretq_u64_u8(mor); + if (TERM_EXPECT((vgetq_lane_u64(mor64, 0) | vgetq_lane_u64(mor64, 1)) != 0, 0)) { + uint64x2_t m0_64 = vreinterpretq_u64_u8(m0); + uint64_t lo0 = vgetq_lane_u64(m0_64, 0), hi0 = vgetq_lane_u64(m0_64, 1); + if (lo0) return i + (term_ctz64(lo0) >> 3); + if (hi0) return i + 8 + (term_ctz64(hi0) >> 3); + + uint64x2_t m1_64 = vreinterpretq_u64_u8(m1); + uint64_t lo1 = vgetq_lane_u64(m1_64, 0), hi1 = vgetq_lane_u64(m1_64, 1); + if (lo1) return i + 16 + (term_ctz64(lo1) >> 3); + if (hi1) return i + 24 + (term_ctz64(hi1) >> 3); + + uint64x2_t m2_64 = vreinterpretq_u64_u8(m2); + uint64_t lo2 = vgetq_lane_u64(m2_64, 0), hi2 = vgetq_lane_u64(m2_64, 1); + if (lo2) return i + 32 + (term_ctz64(lo2) >> 3); + if (hi2) return i + 40 + (term_ctz64(hi2) >> 3); + + uint64x2_t m3_64 = vreinterpretq_u64_u8(m3); + uint64_t lo3 = vgetq_lane_u64(m3_64, 0), hi3 = vgetq_lane_u64(m3_64, 1); + if (lo3) return i + 48 + (term_ctz64(lo3) >> 3); + if (hi3) return i + 56 + (term_ctz64(hi3) >> 3); + } + } + for (; i + 16 <= len; i += 16) { + uint8x16_t v = vld1q_u8((const uint8_t *)buf + i); + uint8x16_t mask = vcgeq_u8(vsubq_u8(v, v20), vlim); + uint64x2_t m64 = vreinterpretq_u64_u8(mask); + uint64_t lo = vgetq_lane_u64(m64, 0); + uint64_t hi64 = vgetq_lane_u64(m64, 1); + if (lo) return i + (term_ctz64(lo) >> 3); + if (hi64) return i + 8 + (term_ctz64(hi64) >> 3); + } + if (i + 8 <= len) { + uint8x8_t v8 = vld1_u8((const uint8_t *)buf + i); + uint8x8_t mask8 = vcge_u8(vsub_u8(v8, vget_low_u8(v20)), vget_low_u8(vlim)); + uint64_t m64 = vget_lane_u64(vreinterpret_u64_u8(mask8), 0); + if (m64) return i + (term_ctz64(m64) >> 3); + i += 8; + } +#elif defined(__x86_64__) + const __m128i v_offset = _mm_set1_epi8((char)(0x20 + 0x80)); + const __m128i v_limit = _mm_set1_epi8((char)(hi - 0xa1)); + for (; i + 64 <= len; i += 64) { + __m128i m0 = _mm_cmpgt_epi8(_mm_sub_epi8(_mm_loadu_si128((const __m128i *)(buf + i)), v_offset), v_limit); + __m128i m1 = _mm_cmpgt_epi8(_mm_sub_epi8(_mm_loadu_si128((const __m128i *)(buf + i + 16)), v_offset), v_limit); + __m128i m2 = _mm_cmpgt_epi8(_mm_sub_epi8(_mm_loadu_si128((const __m128i *)(buf + i + 32)), v_offset), v_limit); + __m128i m3 = _mm_cmpgt_epi8(_mm_sub_epi8(_mm_loadu_si128((const __m128i *)(buf + i + 48)), v_offset), v_limit); + if (TERM_EXPECT(_mm_movemask_epi8(_mm_or_si128(_mm_or_si128(m0, m1), _mm_or_si128(m2, m3))) != 0, 0)) { + uint64_t mask = (uint64_t)(unsigned)_mm_movemask_epi8(m0) | + (uint64_t)(unsigned)_mm_movemask_epi8(m1) << 16 | + (uint64_t)(unsigned)_mm_movemask_epi8(m2) << 32 | + (uint64_t)(unsigned)_mm_movemask_epi8(m3) << 48; + return i + (size_t)term_ctz64(mask); + } + } + for (; i + 16 <= len; i += 16) { + int mask = _mm_movemask_epi8(_mm_cmpgt_epi8( + _mm_sub_epi8(_mm_loadu_si128((const __m128i *)(buf + i)), v_offset), v_limit)); + if (mask != 0) + return i + (size_t)term_ctz32((unsigned)mask); + } +#endif + for (; i < len; i++) { + unsigned char c = (unsigned char)buf[i]; + if (c < 0x20 || c >= hi) + return i; + } + return len; +} + +/* Copy n bytes with loads that stay inside [src, src + n). Short text runs + * are common; a libc call costs more than the copy. */ +static inline TERM_ALWAYS_INLINE void bytes_copy_small(uint8_t *restrict dst, + const char *restrict src, int n) +{ + if (TERM_EXPECT((unsigned)(n - 16) <= 16u, 1)) { + /* 16 to 32 bytes: two overlapping 16-byte copies. Both loads stay + * inside [src, src + n). This is the usual history line length, and + * two copies cost less here than the loop below. */ + memcpy(dst, src, 16); + memcpy(dst + n - 16, src + n - 16, 16); + } else if (n > 256) { + memcpy(dst, src, (size_t)n); + } else if (n >= 16) { + int k = 0; + for (; k + 16 <= n; k += 16) + memcpy(dst + k, src + k, 16); + if (k < n) + memcpy(dst + n - 16, src + n - 16, 16); + } else if (n >= 8) { + uint64_t a, b; + memcpy(&a, src, 8); + memcpy(&b, src + n - 8, 8); + memcpy(dst, &a, 8); + memcpy(dst + n - 8, &b, 8); + } else if (n >= 4) { + uint32_t a, b; + memcpy(&a, src, 4); + memcpy(&b, src + n - 4, 4); + memcpy(dst, &a, 4); + memcpy(dst + n - 4, &b, 4); + } else { + for (int k = 0; k < n; k++) + dst[k] = (uint8_t)src[k]; + } +} + +static inline size_t scan_crlf_tail(const char *buf, size_t i, size_t len) +{ + while (i + 1 < len && buf[i] == '\r' && buf[i + 1] == '\n') + i += 2; + return i; +} + +#if defined(TERM_KERNEL_AVX512) +TERM_TARGET_AVX512 +static inline size_t scan_crlf_pairs_avx512(const char *buf, size_t len) +{ + size_t i = 0; + const __m512i pair = _mm512_set1_epi16(0x0A0D); + for (; i + 64 <= len; i += 64) { + uint64_t eq = _mm512_cmpeq_epi8_mask(_mm512_loadu_si512((const void *)(buf + i)), pair); + if (eq != ~0ull) + return i + ((size_t)term_ctz64(~eq) & ~(size_t)1); + } + return scan_crlf_tail(buf, i, len); +} +#endif + +#if defined(TERM_KERNEL_AVX2) +TERM_TARGET_AVX2 +static inline size_t scan_crlf_pairs_avx2(const char *buf, size_t len) +{ + size_t i = 0; + const __m256i pair = _mm256_set1_epi16(0x0A0D); + for (; i + 32 <= len; i += 32) { + uint32_t eq = (uint32_t)_mm256_movemask_epi8( + _mm256_cmpeq_epi8(_mm256_loadu_si256((const __m256i *)(const void *)(buf + i)), pair)); + if (eq != ~0u) + return i + ((size_t)term_ctz32(~eq) & ~(size_t)1); + } + return scan_crlf_tail(buf, i, len); +} +#endif + +/* The length of the prefix of buf made of "\r\n" pairs; always even. */ +static inline size_t scan_crlf_pairs(const char *buf, size_t len) +{ +#if defined(TERM_KERNEL_AVX512) + if (term_use_avx512()) + return scan_crlf_pairs_avx512(buf, len); +#endif +#if defined(TERM_KERNEL_AVX2) + if (term_use_avx2()) + return scan_crlf_pairs_avx2(buf, len); +#endif + size_t i = 0; +#if defined(__aarch64__) + const uint8x16_t pair = vreinterpretq_u8_u16(vdupq_n_u16(0x0A0D)); + for (; i + 16 <= len; i += 16) { + uint8x16_t eq = vceqq_u8(vld1q_u8((const uint8_t *)buf + i), pair); + uint16_t bits = term_lane_bits(eq); + if (bits != 0xFFFF) + return i + ((size_t)term_ctz32((unsigned)~bits & 0xFFFFu) & ~(size_t)1); + } +#endif + return scan_crlf_tail(buf, i, len); +} + +#if defined(TERM_KERNEL_AVX512) +TERM_TARGET_AVX512 +static inline size_t scan_byte_run_avx512(const char *buf, size_t len, char byte) +{ + size_t i = 0; + const __m512i want = _mm512_set1_epi8(byte); + for (; i + 64 <= len; i += 64) { + uint64_t eq = _mm512_cmpeq_epi8_mask(_mm512_loadu_si512((const void *)(buf + i)), want); + if (eq != ~0ull) + return i + (size_t)term_ctz64(~eq); + } + while (i < len && buf[i] == byte) + i++; + return i; +} +#endif + +#if defined(TERM_KERNEL_AVX2) +TERM_TARGET_AVX2 +static inline size_t scan_byte_run_avx2(const char *buf, size_t len, char byte) +{ + size_t i = 0; + const __m256i want = _mm256_set1_epi8(byte); + for (; i + 32 <= len; i += 32) { + uint32_t eq = (uint32_t)_mm256_movemask_epi8( + _mm256_cmpeq_epi8(_mm256_loadu_si256((const __m256i *)(const void *)(buf + i)), want)); + if (eq != ~0u) + return i + (size_t)term_ctz32(~eq); + } + while (i < len && buf[i] == byte) + i++; + return i; +} +#endif + +/* The length of the prefix of buf made of byte. */ +static inline size_t scan_byte_run(const char *buf, size_t len, char byte) +{ +#if defined(TERM_KERNEL_AVX512) + if (term_use_avx512()) + return scan_byte_run_avx512(buf, len, byte); +#endif +#if defined(TERM_KERNEL_AVX2) + if (term_use_avx2()) + return scan_byte_run_avx2(buf, len, byte); +#endif + size_t i = 0; +#if defined(__aarch64__) + const uint8x16_t want = vdupq_n_u8((uint8_t)byte); + for (; i + 16 <= len; i += 16) { + uint16_t bits = term_lane_bits(vceqq_u8(vld1q_u8((const uint8_t *)buf + i), want)); + if (bits != 0xFFFF) + return i + (size_t)term_ctz32((unsigned)~bits & 0xFFFFu); + } +#elif defined(__x86_64__) + const __m128i want = _mm_set1_epi8(byte); + for (; i + 16 <= len; i += 16) { + unsigned eq = (unsigned)_mm_movemask_epi8( + _mm_cmpeq_epi8(_mm_loadu_si128((const __m128i *)(const void *)(buf + i)), want)); + if (eq != 0xFFFFu) + return i + (size_t)term_ctz32(~eq & 0xFFFFu); + } +#endif + while (i < len && buf[i] == byte) + i++; + return i; +} + +static inline int scan_ascii_run(const char *buf, int len) +{ + return (int)scan_printable_span(buf, (size_t)len, 0x7f); +} + +static inline int scan_ascii_lf_tail(const char *buf, int i, int len, uint16_t *ends, int count) +{ + for (; i < len; i++) { + unsigned char ch = (unsigned char)buf[i]; + if (ch == '\n') ends[++count] = (uint16_t)(i + 1); + else if (ch - 0x20u >= 0x5fu) break; + } + return count; +} + +#if defined(TERM_KERNEL_AVX512) +TERM_TARGET_AVX512 TERM_MAYBE_UNUSED +static int scan_ascii_lf_avx512(const char *buf, int len, uint16_t *ends) +{ + int i = 0, count = 0; + ends[0] = 0; + const __m512i lanes = _mm512_setr_epi32( + 0x03020100,0x07060504,0x0b0a0908,0x0f0e0d0c, + 0x13121110,0x17161514,0x1b1a1918,0x1f1e1d1c, + 0x23222120,0x27262524,0x2b2a2928,0x2f2e2d2c, + 0x33323130,0x37363534,0x3b3a3938,0x3f3e3d3c); + __m128i block = _mm_set1_epi16(1); + for (; i + 256 <= len; i += 256) { + __m512i v[4]; + uint64_t nl[4]; + uint64_t bad = 0; + for (int q = 0; q < 4; q++) { + v[q] = _mm512_loadu_si512(buf + i + q * 64); + nl[q] = _mm512_cmpeq_epi8_mask(v[q], _mm512_set1_epi8('\n')); + bad |= _mm512_cmpge_epu8_mask(_mm512_sub_epi8(v[q], _mm512_set1_epi8(0x20)), + _mm512_set1_epi8(0x5f)) & ~nl[q]; + } + if (bad) break; + for (int q = 0; q < 4; q++) { + unsigned n = (unsigned)term_popcount64(nl[q]); + __m512i packed = _mm512_maskz_compress_epi8(nl[q], lanes); + __m128i offsets = _mm_add_epi16(_mm_cvtepu8_epi16(_mm512_castsi512_si128(packed)), block); + _mm_storeu_si128((__m128i *)(ends + count + 1), offsets); + if (TERM_EXPECT(n > 8, 0)) { + __m512i base = _mm512_broadcast_i32x4(block); + __m512i wide = _mm512_add_epi16(_mm512_cvtepu8_epi16(_mm512_castsi512_si256(packed)), base); + _mm512_storeu_si512(ends + count + 1, wide); + if (n > 32) { + wide = _mm512_add_epi16(_mm512_cvtepu8_epi16(_mm512_extracti64x4_epi64(packed, 1)), base); + _mm512_storeu_si512(ends + count + 33, wide); + } + } + count += (int)n; + block = _mm_add_epi16(block, _mm_set1_epi16(64)); + } + } + for (; i + 64 <= len; i += 64) { + __m512i v = _mm512_loadu_si512(buf + i); + uint64_t nl = _mm512_cmpeq_epi8_mask(v, _mm512_set1_epi8('\n')); + uint64_t bad = _mm512_cmpge_epu8_mask( + _mm512_sub_epi8(v, _mm512_set1_epi8(0x20)), _mm512_set1_epi8(0x5f)) & ~nl; + if (bad) break; + unsigned n = (unsigned)term_popcount64(nl); + __m512i packed = _mm512_maskz_compress_epi8(nl, lanes); + __m128i offsets = _mm_add_epi16(_mm_cvtepu8_epi16(_mm512_castsi512_si128(packed)), block); + _mm_storeu_si128((__m128i *)(ends + count + 1), offsets); + if (TERM_EXPECT(n > 8, 0)) { + __m512i base = _mm512_broadcast_i32x4(block); + __m512i wide = _mm512_add_epi16(_mm512_cvtepu8_epi16(_mm512_castsi512_si256(packed)), base); + _mm512_storeu_si512(ends + count + 1, wide); + if (n > 32) { + wide = _mm512_add_epi16(_mm512_cvtepu8_epi16(_mm512_extracti64x4_epi64(packed, 1)), base); + _mm512_storeu_si512(ends + count + 33, wide); + } + } + count += (int)n; + block = _mm_add_epi16(block, _mm_set1_epi16(64)); + } + return scan_ascii_lf_tail(buf, i, len, ends, count); +} +#endif + +#if defined(TERM_KERNEL_AVX2) +TERM_TARGET_AVX2 TERM_MAYBE_UNUSED +static int scan_ascii_lf_avx2(const char *buf, int len, uint16_t *ends) +{ + int i = 0, count = 0; + ends[0] = 0; + for (; i + 32 <= len; i += 32) { + __m256i v = _mm256_loadu_si256((const __m256i *)(buf + i)); + uint32_t nl = (uint32_t)_mm256_movemask_epi8(_mm256_cmpeq_epi8(v, _mm256_set1_epi8('\n'))); + __m256i d = _mm256_sub_epi8(v, _mm256_set1_epi8((char)0xa0)); + uint32_t bad = (uint32_t)_mm256_movemask_epi8( + _mm256_cmpgt_epi8(d, _mm256_set1_epi8((char)0xde))) & ~nl; + if (bad) break; + while (nl) { + ends[++count] = (uint16_t)(i + term_ctz32(nl) + 1); + nl &= nl - 1; + } + } + return scan_ascii_lf_tail(buf, i, len, ends, count); +} +#endif + +/* Find complete ASCII lines. Stop at the first byte other than text or LF. + * The caller supplies len + 65 entries. len must fit in uint16_t. */ +static TERM_NOINLINE TERM_MAYBE_UNUSED int scan_ascii_lf(const char *buf, int len, uint16_t *ends) +{ +#if defined(TERM_KERNEL_AVX512) + if (term_use_avx512()) + return scan_ascii_lf_avx512(buf, len, ends); +#endif +#if defined(TERM_KERNEL_AVX2) + if (term_use_avx2()) + return scan_ascii_lf_avx2(buf, len, ends); +#endif + int i = 0, count = 0; + ends[0] = 0; +#if defined(__aarch64__) + for (; i + 32 <= len; i += 32) { + uint32_t nl = 0, bad = 0; + for (int q = 0; q < 2; q++) { + uint8x16_t v = vld1q_u8((const uint8_t *)buf + i + q * 16); + nl |= (uint32_t)term_lane_bits(vceqq_u8(v, vdupq_n_u8('\n'))) << (q * 16); + bad |= (uint32_t)term_lane_bits(vcgeq_u8(vsubq_u8(v, vdupq_n_u8(0x20)), + vdupq_n_u8(0x5f))) << (q * 16); + } + bad &= ~nl; + if (bad) break; + while (nl) { + ends[++count] = (uint16_t)(i + term_ctz32(nl) + 1); + nl &= nl - 1; + } + } +#endif + return scan_ascii_lf_tail(buf, i, len, ends, count); +} + +#if defined(TERM_KERNEL_AVX512) +TERM_TARGET_AVX512 +static inline void fill_cells_avx512(struct cell * restrict row, int cols, uint32_t fg, uint32_t bg) +{ + __m128i cell = _mm_setr_epi32(' ', (int)fg, (int)bg, 0); + __m512i blank = _mm512_broadcast_i32x4(cell); + int c = 0; + for (; c + 8 <= cols; c += 8) { + _mm512_storeu_si512((void *)(row + c), blank); + _mm512_storeu_si512((void *)(row + c + 4), blank); + } + int rem = cols - c; + if (rem >= 4) { + _mm512_storeu_si512((void *)(row + c), blank); + c += 4; + rem -= 4; + } + if (rem > 0) { + uint16_t store_mask = (uint16_t)((1u << (4 * rem)) - 1); + _mm512_mask_storeu_epi32((void *)(row + c), store_mask, blank); + } +} +#endif + +#if defined(TERM_KERNEL_AVX2) +TERM_TARGET_AVX2 +static inline void fill_cells_avx2(struct cell * restrict row, int cols, uint32_t fg, uint32_t bg) +{ + struct cell fill = { .c = ' ', .fg = fg, .bg = bg, .flags = 0, .pad = 0 }; + __m256i blank = _mm256_setr_epi32(' ', (int)fg, (int)bg, 0, ' ', (int)fg, (int)bg, 0); + int c = 0; + for (; c + 8 <= cols; c += 8) { + char *d = (char *)(row + c); + _mm256_storeu_si256((__m256i *)(d), blank); + _mm256_storeu_si256((__m256i *)(d + 32), blank); + _mm256_storeu_si256((__m256i *)(d + 64), blank); + _mm256_storeu_si256((__m256i *)(d + 96), blank); + } + for (; c < cols; c++) { + row[c] = fill; + } +} +#endif + +static inline void fill_cells(struct cell * restrict row, int cols, uint32_t fg, uint32_t bg) +{ + struct cell fill = { .c = ' ', .fg = fg, .bg = bg, .flags = 0, .pad = 0 }; +#if defined(TERM_KERNEL_AVX512) + if (term_use_avx512()) { + if (TERM_EXPECT(cols > 0, 1)) + fill_cells_avx512(row, cols, fg, bg); + return; + } +#endif +#if defined(TERM_KERNEL_AVX2) + if (term_use_avx2() && TERM_EXPECT(cols >= 8, 1)) { + fill_cells_avx2(row, cols, fg, bg); + return; + } +#endif +#if defined(__aarch64__) + if (TERM_EXPECT(cols > 0, 1)) { + uint32x4_t blank = { ' ', fg, bg, 0 }; + uint32x4x4_t b4 = {{ blank, blank, blank, blank }}; + int c = 0; + for (; c + 8 <= cols; c += 8) { + vst1q_u32_x4((uint32_t *)(void *)(row + c), b4); + vst1q_u32_x4((uint32_t *)(void *)(row + c + 4), b4); + } + for (; c + 4 <= cols; c += 4) { + vst1q_u32_x4((uint32_t *)(void *)(row + c), b4); + } + for (; c < cols; c++) { + vst1q_u32((uint32_t *)(void *)(row + c), blank); + } + return; + } +#elif defined(__x86_64__) + { + __m128i blank = _mm_setr_epi32(' ', (int)fg, (int)bg, 0); + int c = 0; + for (; c + 4 <= cols; c += 4) { + _mm_storeu_si128((__m128i *)(void *)(row + c), blank); + _mm_storeu_si128((__m128i *)(void *)(row + c + 1), blank); + _mm_storeu_si128((__m128i *)(void *)(row + c + 2), blank); + _mm_storeu_si128((__m128i *)(void *)(row + c + 3), blank); + } + for (; c < cols; c++) + _mm_storeu_si128((__m128i *)(void *)(row + c), blank); + return; + } +#endif + for (int c = 0; c < cols; c++) { + row[c] = fill; + } +} + +static inline void clear_cells_default(struct cell *row, int cols) +{ + fill_cells(row, cols, TERM_COLOR_FG, TERM_COLOR_BG); +} + +#if defined(TERM_KERNEL_AVX512) +TERM_TARGET_AVX512 +static inline void put_ascii_chunk_avx512(struct cell * restrict dst, const char * restrict src, int n, + uint32_t fg, uint32_t bg, uint32_t flags) +{ + int k = 0; + __m128i cell = _mm_setr_epi32(0, (int)fg, (int)bg, (int)flags); + __m512i base = _mm512_broadcast_i32x4(cell); + const __m512i idx0 = _mm512_setr_epi32(0, 0, 0, 0, 1, 0, 0, 0, 2, 0, 0, 0, 3, 0, 0, 0); + const __m512i idx1 = _mm512_setr_epi32(4, 0, 0, 0, 5, 0, 0, 0, 6, 0, 0, 0, 7, 0, 0, 0); + const __m512i idx2 = _mm512_setr_epi32(8, 0, 0, 0, 9, 0, 0, 0, 10, 0, 0, 0, 11, 0, 0, 0); + const __m512i idx3 = _mm512_setr_epi32(12, 0, 0, 0, 13, 0, 0, 0, 14, 0, 0, 0, 15, 0, 0, 0); + + for (; k + 16 <= n; k += 16) { + __m512i cps = _mm512_cvtepu8_epi32(_mm_loadu_si128((const __m128i *)(src + k))); + _mm512_storeu_si512(dst + k + 0, _mm512_mask_permutexvar_epi32(base, 0x1111, idx0, cps)); + _mm512_storeu_si512(dst + k + 4, _mm512_mask_permutexvar_epi32(base, 0x1111, idx1, cps)); + _mm512_storeu_si512(dst + k + 8, _mm512_mask_permutexvar_epi32(base, 0x1111, idx2, cps)); + _mm512_storeu_si512(dst + k + 12, _mm512_mask_permutexvar_epi32(base, 0x1111, idx3, cps)); + } + int rem = n - k; + if (rem > 0) { + __m128i v = _mm_maskz_loadu_epi8((1 << rem) - 1, src + k); + __m512i cps = _mm512_cvtepu8_epi32(v); + if (rem <= 4) { + uint16_t m = (uint16_t)((1u << (4 * rem)) - 1); + _mm512_mask_storeu_epi32(dst + k, m, _mm512_mask_permutexvar_epi32(base, 0x1111, idx0, cps)); + return; + } + _mm512_storeu_si512(dst + k, _mm512_mask_permutexvar_epi32(base, 0x1111, idx0, cps)); + if (rem <= 8) { + uint16_t m = (uint16_t)((1u << (4 * (rem - 4))) - 1); + _mm512_mask_storeu_epi32(dst + k + 4, m, _mm512_mask_permutexvar_epi32(base, 0x1111, idx1, cps)); + return; + } + _mm512_storeu_si512(dst + k + 4, _mm512_mask_permutexvar_epi32(base, 0x1111, idx1, cps)); + if (rem <= 12) { + uint16_t m = (uint16_t)((1u << (4 * (rem - 8))) - 1); + _mm512_mask_storeu_epi32(dst + k + 8, m, _mm512_mask_permutexvar_epi32(base, 0x1111, idx2, cps)); + return; + } + _mm512_storeu_si512(dst + k + 8, _mm512_mask_permutexvar_epi32(base, 0x1111, idx2, cps)); + uint16_t m = (uint16_t)((1u << (4 * (rem - 12))) - 1); + _mm512_mask_storeu_epi32(dst + k + 12, m, _mm512_mask_permutexvar_epi32(base, 0x1111, idx3, cps)); + } +} +#endif + +#if defined(TERM_KERNEL_AVX2) +TERM_TARGET_AVX2 +static inline void put_ascii_chunk_avx2(struct cell * restrict dst, const char * restrict src, int n, + uint32_t fg, uint32_t bg, uint32_t flags) +{ + int k = 0; + __m128i cell = _mm_setr_epi32(0, (int)fg, (int)bg, (int)flags); + __m256i base = _mm256_broadcastsi128_si256(cell); +#define ASCII_PAIR(o, q) _mm256_storeu_si256((__m256i *)(void *)(dst + (o) + 2 * (q)), \ + _mm256_blend_epi32(_mm256_permutevar8x32_epi32(cps, \ + _mm256_setr_epi32(2*(q), 0, 0, 0, 2*(q)+1, 0, 0, 0)), base, 0xee)) +#define ASCII_8(o) do { \ + __m256i cps = _mm256_cvtepu8_epi32(_mm_loadl_epi64((const __m128i *)(void *)(src + (o)))); \ + ASCII_PAIR(o, 0); ASCII_PAIR(o, 1); ASCII_PAIR(o, 2); ASCII_PAIR(o, 3); \ + } while (0) + for (; k + 16 <= n; k += 16) { + ASCII_8(k); + ASCII_8(k + 8); + } + if (k + 8 <= n) { + ASCII_8(k); + k += 8; + } + if (k < n && n >= 8) { + ASCII_8(n - 8); + k = n; + } +#undef ASCII_8 +#undef ASCII_PAIR + for (; k < n; k++) + _mm_storeu_si128((__m128i *)(void *)(dst + k), + _mm_or_si128(cell, _mm_cvtsi32_si128((unsigned char)src[k]))); +} +#endif + +TERM_ALWAYS_INLINE +static inline void put_ascii_chunk(struct cell * restrict dst, const char * restrict src, int n, + uint32_t fg, uint32_t bg, uint32_t flags) +{ + /* A short chunk costs less inline than as a call. */ +#if defined(TERM_KERNEL_AVX512) + if (n >= 16 && term_use_avx512()) { + put_ascii_chunk_avx512(dst, src, n, fg, bg, flags); + return; + } +#endif +#if defined(TERM_KERNEL_AVX2) + if (n >= 16 && term_use_avx2()) { + put_ascii_chunk_avx2(dst, src, n, fg, bg, flags); + return; + } +#endif + int k = 0; +#if defined(__aarch64__) + uint32x4_t fg4 = vdupq_n_u32(fg); + uint32x4_t bg_flags = vcombine_u32(vcreate_u32(((uint64_t)flags << 32) | bg), + vcreate_u32(((uint64_t)flags << 32) | bg)); + for (; k + 8 <= n; k += 8) { + uint16x8_t cp16 = vmovl_u8(vld1_u8((const uint8_t *)src + k)); + uint32x4_t cp_lo = vmovl_u16(vget_low_u16(cp16)); + uint32x4_t cp_hi = vmovl_u16(vget_high_u16(cp16)); + + uint32x4_t c_fg_01 = vzip1q_u32(cp_lo, fg4); + uint32x4_t c_fg_23 = vzip2q_u32(cp_lo, fg4); + uint32x4_t c_fg_45 = vzip1q_u32(cp_hi, fg4); + uint32x4_t c_fg_67 = vzip2q_u32(cp_hi, fg4); + + uint32x4_t cell0 = (uint32x4_t)vzip1q_u64((uint64x2_t)c_fg_01, (uint64x2_t)bg_flags); + uint32x4_t cell1 = (uint32x4_t)vzip2q_u64((uint64x2_t)c_fg_01, (uint64x2_t)bg_flags); + uint32x4_t cell2 = (uint32x4_t)vzip1q_u64((uint64x2_t)c_fg_23, (uint64x2_t)bg_flags); + uint32x4_t cell3 = (uint32x4_t)vzip2q_u64((uint64x2_t)c_fg_23, (uint64x2_t)bg_flags); + uint32x4_t cell4 = (uint32x4_t)vzip1q_u64((uint64x2_t)c_fg_45, (uint64x2_t)bg_flags); + uint32x4_t cell5 = (uint32x4_t)vzip2q_u64((uint64x2_t)c_fg_45, (uint64x2_t)bg_flags); + uint32x4_t cell6 = (uint32x4_t)vzip1q_u64((uint64x2_t)c_fg_67, (uint64x2_t)bg_flags); + uint32x4_t cell7 = (uint32x4_t)vzip2q_u64((uint64x2_t)c_fg_67, (uint64x2_t)bg_flags); + + uint32x4x4_t q0 = {{ cell0, cell1, cell2, cell3 }}; + uint32x4x4_t q1 = {{ cell4, cell5, cell6, cell7 }}; + vst1q_u32_x4((uint32_t *)(void *)(dst + k + 0), q0); + vst1q_u32_x4((uint32_t *)(void *)(dst + k + 4), q1); + } + for (; k + 4 <= n; k += 4) { + uint32_t bytes; + memcpy(&bytes, src + k, sizeof(bytes)); + uint16x4_t cp16 = vget_low_u16(vmovl_u8(vreinterpret_u8_u32(vdup_n_u32(bytes)))); + uint32x4_t cp_lo = vmovl_u16(cp16); + uint32x4_t c_fg_01 = vzip1q_u32(cp_lo, fg4); + uint32x4_t c_fg_23 = vzip2q_u32(cp_lo, fg4); + uint32x4_t cell0 = (uint32x4_t)vzip1q_u64((uint64x2_t)c_fg_01, (uint64x2_t)bg_flags); + uint32x4_t cell1 = (uint32x4_t)vzip2q_u64((uint64x2_t)c_fg_01, (uint64x2_t)bg_flags); + uint32x4_t cell2 = (uint32x4_t)vzip1q_u64((uint64x2_t)c_fg_23, (uint64x2_t)bg_flags); + uint32x4_t cell3 = (uint32x4_t)vzip2q_u64((uint64x2_t)c_fg_23, (uint64x2_t)bg_flags); + uint32x4x4_t q0 = {{ cell0, cell1, cell2, cell3 }}; + vst1q_u32_x4((uint32_t *)(void *)(dst + k + 0), q0); + } + if (k < n && n >= 4) { + uint32_t bytes; + memcpy(&bytes, src + n - 4, sizeof(bytes)); + uint16x4_t cp16 = vget_low_u16(vmovl_u8(vreinterpret_u8_u32(vdup_n_u32(bytes)))); + uint32x4_t cp_lo = vmovl_u16(cp16); + uint32x4_t c_fg_01 = vzip1q_u32(cp_lo, fg4); + uint32x4_t c_fg_23 = vzip2q_u32(cp_lo, fg4); + uint32x4_t cell0 = (uint32x4_t)vzip1q_u64((uint64x2_t)c_fg_01, (uint64x2_t)bg_flags); + uint32x4_t cell1 = (uint32x4_t)vzip2q_u64((uint64x2_t)c_fg_01, (uint64x2_t)bg_flags); + uint32x4_t cell2 = (uint32x4_t)vzip1q_u64((uint64x2_t)c_fg_23, (uint64x2_t)bg_flags); + uint32x4_t cell3 = (uint32x4_t)vzip2q_u64((uint64x2_t)c_fg_23, (uint64x2_t)bg_flags); + uint32x4x4_t q0 = {{ cell0, cell1, cell2, cell3 }}; + vst1q_u32_x4((uint32_t *)(void *)(dst + n - 4), q0); + k = n; + } +#elif defined(__x86_64__) + const __m128i zero = _mm_setzero_si128(); + const __m128i fg4 = _mm_set1_epi32((int)fg); + const __m128i bg_flags = _mm_setr_epi32((int)bg, (int)flags, (int)bg, (int)flags); +#define ASCII_4(o, cp32) do { \ + __m128i c_fg_01 = _mm_unpacklo_epi32((cp32), fg4); \ + __m128i c_fg_23 = _mm_unpackhi_epi32((cp32), fg4); \ + _mm_storeu_si128((__m128i *)(void *)(dst + (o) + 0), _mm_unpacklo_epi64(c_fg_01, bg_flags)); \ + _mm_storeu_si128((__m128i *)(void *)(dst + (o) + 1), _mm_unpackhi_epi64(c_fg_01, bg_flags)); \ + _mm_storeu_si128((__m128i *)(void *)(dst + (o) + 2), _mm_unpacklo_epi64(c_fg_23, bg_flags)); \ + _mm_storeu_si128((__m128i *)(void *)(dst + (o) + 3), _mm_unpackhi_epi64(c_fg_23, bg_flags)); \ + } while (0) + for (; k + 8 <= n; k += 8) { + __m128i cp16 = _mm_unpacklo_epi8(_mm_loadl_epi64((const __m128i *)(const void *)(src + k)), zero); + ASCII_4(k, _mm_unpacklo_epi16(cp16, zero)); + ASCII_4(k + 4, _mm_unpackhi_epi16(cp16, zero)); + } +#define ASCII_4_AT(o) do { \ + uint32_t bytes; \ + memcpy(&bytes, src + (o), sizeof(bytes)); \ + __m128i cp16 = _mm_unpacklo_epi8(_mm_cvtsi32_si128((int)bytes), zero); \ + ASCII_4(o, _mm_unpacklo_epi16(cp16, zero)); \ + } while (0) + if (k + 4 <= n) { + ASCII_4_AT(k); + k += 4; + } + if (k < n && n >= 4) { + ASCII_4_AT(n - 4); + k = n; + } +#undef ASCII_4_AT +#undef ASCII_4 +#endif + for (; k < n; k++) { + dst[k] = (struct cell){ + .c = (unsigned char)src[k], + .fg = fg, + .bg = bg, + .flags = (uint16_t)flags, + .pad = (uint16_t)(flags >> 16), + }; + } +} + +#endif diff --git a/src/meson.build b/src/meson.build new file mode 100644 index 0000000..6eaf0d4 --- /dev/null +++ b/src/meson.build @@ -0,0 +1,27 @@ +subdir('osc') + +# The x86-64 SIMD kernels pick their level at run time (see simd.h), so the +# library needs no ISA flags. -march in c_args sets the lowest level. +term_headless_lib = both_libraries('term_headless', + 'core.c', + 'escape.c', + 'fastcsi.c', + 'gstore.c', + 'hist.c', + 'hist_view.c', + 'input.c', + 'isa.c', + 'palette.c', + 'scan.c', + 'screen.c', + 'selection.c', + 'snapshot.c', + 'term.c', + 'text.c', + 'text_run.c', + term_osc_src, + include_directories: term_private_inc, + dependencies: [m_dep], + gnu_symbol_visibility: 'hidden', + install: false +) diff --git a/src/osc/clipboard.c b/src/osc/clipboard.c new file mode 100644 index 0000000..a5ac59d --- /dev/null +++ b/src/osc/clipboard.c @@ -0,0 +1,76 @@ +#include "osc.h" +#include "../internal.h" +#include <stdlib.h> +#include <string.h> + +static int b64_val(char c) +{ + if (c >= 'A' && c <= 'Z') return c - 'A'; + if (c >= 'a' && c <= 'z') return c - 'a' + 26; + if (c >= '0' && c <= '9') return c - '0' + 52; + if (c == '+') return 62; + if (c == '/') return 63; + return -1; +} + +static bool b64_decode(const char *s, size_t n, char *out, size_t *out_len) +{ + size_t pad = 0; + while (pad < n && s[n - pad - 1] == '=') + pad++; + n -= pad; + if (pad > 2 || (pad && n % 4 + pad != 4)) + return false; + size_t rem = n % 4; + if (rem == 1) + return false; + size_t i = 0, o = 0; + for (; i + 4 <= n; i += 4) { + int a = b64_val(s[i]), b = b64_val(s[i + 1]); + int c = b64_val(s[i + 2]), d = b64_val(s[i + 3]); + if ((a | b | c | d) < 0) + return false; + out[o++] = (char)(a << 2 | b >> 4); + out[o++] = (char)(b << 4 | c >> 2); + out[o++] = (char)(c << 6 | d); + } + if (rem >= 2) { + int a = b64_val(s[i]), b = b64_val(s[i + 1]); + if ((a | b) < 0) + return false; + out[o++] = (char)(a << 2 | b >> 4); + if (rem == 3) { + int c = b64_val(s[i + 2]); + if (c < 0) + return false; + out[o++] = (char)(b << 4 | c >> 2); + } + } + *out_len = o; + return true; +} + +void term_osc52(struct term *t, const char *params) +{ + const char *sep = strchr(params, ';'); + if (!sep || sep == params) + return; + const char *pd = sep + 1; + if (!strcmp(pd, "?") || !t->cb.clipboard) + return; + size_t sel_len = (size_t)(sep - params); + char sel[64]; + if (sel_len >= sizeof(sel)) + return; + memcpy(sel, params, sel_len); + sel[sel_len] = '\0'; + if (!*pd) { + t->cb.clipboard(t->cb.ud, sel, "", 0); + return; + } + size_t enc_len = strlen(pd); + char *dec = (char *)pd; + size_t dec_len; + if (b64_decode(pd, enc_len, dec, &dec_len)) + t->cb.clipboard(t->cb.ud, sel, dec, dec_len); +} diff --git a/src/osc/dispatch.c b/src/osc/dispatch.c new file mode 100644 index 0000000..4cf986e --- /dev/null +++ b/src/osc/dispatch.c @@ -0,0 +1,227 @@ +#include "osc.h" +#include "../internal.h" +#include "../palette.h" +#include "../screen.h" +#include <stdlib.h> +#include <string.h> +#include <stdio.h> + +#define OSC_HEAP_INIT 4096 +#define OSC_HEAP_KEEP (1u << 20) + +size_t term_osc_room(const struct term *t) +{ + size_t cap = t->osc_heap ? t->osc_heap_cap : sizeof(t->osc_buf); + return cap - 1 - (size_t)t->osc_len; +} + +char *term_osc_tail(struct term *t) +{ + return (t->osc_heap ? t->osc_heap : t->osc_buf) + t->osc_len; +} + +void term_osc_commit(struct term *t, size_t n) +{ + t->osc_len += (int)n; +} + +char *term_osc_data(struct term *t) +{ + return t->osc_heap ? t->osc_heap : t->osc_buf; +} + +bool term_osc_grow(struct term *t) +{ + if (!t->osc_heap) { + char *p = malloc(OSC_HEAP_INIT); + if (!p) return false; + memcpy(p, t->osc_buf, (size_t)t->osc_len); + t->osc_heap = p; + t->osc_heap_cap = OSC_HEAP_INIT; + return true; + } + if (t->osc_heap_cap >= TERM_OSC_MAX) return false; + size_t cap = t->osc_heap_cap * 2; + if (cap > TERM_OSC_MAX) cap = TERM_OSC_MAX; + char *p = realloc(t->osc_heap, cap); + if (!p) return false; + t->osc_heap = p; + t->osc_heap_cap = cap; + return true; +} + +void term_osc_cleanup(struct term *t) +{ + free(t->osc_heap); + t->osc_heap = NULL; + t->osc_heap_cap = 0; + free(t->dcs_buf); + t->dcs_buf = NULL; + t->dcs_cap = 0; + term_osc8_cleanup(t); + free(t->cwd); + t->cwd = NULL; + free(t->pointer_shape); + t->pointer_shape = NULL; +} + +/* Reply to a color query. An OSC 4 reply names the index it answers for; + * index is negative for OSC 10, 11, and 12. */ +static void send_color_query_reply(struct term *t, int code, int index, uint32_t argb) +{ + char resp[64], prefix[16] = ""; + uint8_t r = (argb >> 16) & 0xff, g = (argb >> 8) & 0xff, b = argb & 0xff; + if (index >= 0) + snprintf(prefix, sizeof(prefix), "%d;", index); + snprintf(resp, sizeof(resp), + "\x1b]%d;%srgb:%02x%02x/%02x%02x/%02x%02x\x1b\\", + code, prefix, r, r, g, g, b, b); + send_response(t, resp); +} + +static void osc_color_set_or_query(struct term *t, int code, const char *params, int slot) +{ + if (!strcmp(params, "?")) { + send_color_query_reply(t, code, -1, t->palette.colors[slot]); + return; + } + uint32_t argb; + if (term_color_parse(params, &argb)) + term_palette_set(&t->palette, slot, argb); +} + +static long osc_next_index(const char **pp) +{ + char *end; + long idx = strtol(*pp, &end, 10); + if (end == *pp || (*end != ';' && *end != '\0')) + return -1; + *pp = end; + if (**pp == ';') (*pp)++; + return idx; +} + +static void osc4_handle(struct term *t, const char *params) +{ + const char *p = params; + while (*p) { + long idx = osc_next_index(&p); + if (idx < 0) + return; + + const char *spec = p; + const char *spec_end = spec; + while (*spec_end && *spec_end != ';') + spec_end++; + size_t spec_len = (size_t)(spec_end - spec); + if (spec_len == 1 && spec[0] == '?') { + if (idx <= 255) + send_color_query_reply(t, 4, (int)idx, t->palette.colors[idx]); + } else { + char buf[128]; + if (idx <= 255 && spec_len < sizeof(buf)) { + memcpy(buf, spec, spec_len); + buf[spec_len] = '\0'; + uint32_t argb; + if (term_color_parse(buf, &argb)) + term_palette_set(&t->palette, (int)idx, argb); + } + } + p = spec_end; + if (*p == ';') p++; + } +} + +static void osc104_handle(struct term *t, const char *params) +{ + if (!*params) { + for (int i = 0; i < 256; i++) + term_palette_reset(&t->palette, i); + return; + } + const char *p = params; + while (*p) { + long idx = osc_next_index(&p); + if (idx < 0) + return; + if (idx <= 255) + term_palette_reset(&t->palette, (int)idx); + } +} + +void term_osc_finish(struct term *t) +{ + if (!term_str_was_truncated(t)) { + char *buf = term_osc_data(t); + buf[t->osc_len] = '\0'; + + if (!t->alternate && t->osc_len >= 5 && !strncmp(buf, "133;", 4) && + (buf[5] == ';' || buf[5] == '\0')) { + if (buf[4] == 'A') { + t->prompt_y = t->cursor_y; + t->has_prompt = true; + } + if (buf[4] == 'C' || buf[4] == 'D') { + t->prompt_y = -1; + t->has_prompt = false; + } + } + + char *rest; + long code = strtol(buf, &rest, 10); + bool no_code = rest == buf; + bool legacy_title = no_code && !strchr(buf, ';'); + if (no_code) { + code = legacy_title || *buf == ';' ? 0 : -1; + rest = buf; + } + if (code >= 0 && (legacy_title || *rest == ';' || *rest == '\0' || code <= 2)) { + const char *params = *rest == ';' ? rest + 1 : rest; + switch (code) { + case 0: case 1: case 2: + if (t->cb.title) + t->cb.title(t->cb.ud, params); + break; + case 4: + osc4_handle(t, params); + break; + case 8: + term_osc8(t, params); + break; + case 10: + osc_color_set_or_query(t, 10, params, TERM_PALETTE_FG); + break; + case 11: + osc_color_set_or_query(t, 11, params, TERM_PALETTE_BG); + break; + case 12: + osc_color_set_or_query(t, 12, params, TERM_PALETTE_CURSOR); + break; + case 52: + term_osc52(t, params); + break; + case 104: + osc104_handle(t, params); + break; + case 110: + term_palette_reset(&t->palette, TERM_PALETTE_FG); + break; + case 111: + term_palette_reset(&t->palette, TERM_PALETTE_BG); + break; + case 112: + term_palette_reset(&t->palette, TERM_PALETTE_CURSOR); + break; + default: + term_osc_misc_dispatch(t, code, params); + break; + } + } + } + t->osc_len = 0; + if (t->osc_heap && t->osc_heap_cap > OSC_HEAP_KEEP) { + free(t->osc_heap); + t->osc_heap = NULL; + t->osc_heap_cap = 0; + } +} diff --git a/src/osc/link.c b/src/osc/link.c new file mode 100644 index 0000000..f31a750 --- /dev/null +++ b/src/osc/link.c @@ -0,0 +1,260 @@ +#include "osc.h" +#include "../internal.h" +#include "../screen.h" +#include <stdlib.h> +#include <string.h> + +#define TERM_LINK_MAX 8192 +#define TERM_LINK_URI_MAX 4096 + +struct link_hash_slot { + uint64_t hash; + uint16_t id; +}; + +struct term_links { + char **uri; + uint16_t *free_ids; + struct link_hash_slot *hash_table; + uint32_t hash_cap; + uint16_t capacity; + uint16_t free_count; + uint16_t next; +}; + +static inline uint64_t hash_uri(const char *s) +{ + uint64_t h = HASH64_BASIS; + while (*s) + h = hash64_step(h, (uint8_t)*s++); + h = hash64_finish(h); + return h ? h : 1; +} + +static void links_insert_hash(struct term_links *l, uint64_t h, uint16_t id) +{ + if (!l->hash_table || !l->hash_cap) + return; + uint32_t mask = l->hash_cap - 1; + uint32_t idx = (uint32_t)h & mask; + for (uint32_t probe = 0; probe <= mask; probe++) { + struct link_hash_slot *slot = &l->hash_table[idx]; + if (slot->hash == 0) { + slot->hash = h; + slot->id = id; + return; + } + idx = (idx + 1) & mask; + } +} + +static bool links_grow(struct term_links *l) +{ + uint16_t new_cap = l->capacity == 0 ? 64 : l->capacity < 4096 ? l->capacity * 2 : TERM_LINK_MAX; + if (new_cap <= l->capacity) + return false; + char **new_uri = realloc(l->uri, sizeof(char *) * new_cap); + if (!new_uri) return false; + uint16_t *new_free = malloc(sizeof(uint16_t) * new_cap); + if (!new_free) { + l->uri = new_uri; + return false; + } + uint32_t new_hash_cap = (uint32_t)new_cap * 2; + struct link_hash_slot *new_hash = calloc(new_hash_cap, sizeof(*new_hash)); + if (!new_hash) { + free(new_free); + l->uri = new_uri; + return false; + } + uint16_t *old_free = l->free_ids; + l->free_ids = new_free; + if (l->capacity) + memcpy(new_free, old_free, sizeof(uint16_t) * l->capacity); + free(old_free); + memset(new_uri + l->capacity, 0, sizeof(char *) * (new_cap - l->capacity)); + l->uri = new_uri; + l->capacity = new_cap; + + free(l->hash_table); + l->hash_table = new_hash; + l->hash_cap = new_hash_cap; + + for (int i = 0; i + 1 < l->next; i++) { + if (l->uri[i]) + links_insert_hash(l, hash_uri(l->uri[i]), (uint16_t)(i + 1)); + } + return true; +} + +static void links_mark_row(const struct cell *row, int len, uint64_t *mark, uint16_t cap) +{ + if (len > TERM_MAX_COLS) len = TERM_MAX_COLS; + for (int i = 0; i < len; i++) { + uint16_t id = row[i].pad; + if (id && id <= cap) + mark[(id - 1) >> 6] |= 1ull << ((id - 1) & 63); + } +} + +static void links_mark_id(uint16_t id, uint64_t *mark, uint16_t cap) +{ + if (id && id <= cap) + mark[(id - 1) >> 6] |= 1ull << ((id - 1) & 63); +} + +struct links_mark { + uint64_t *mark; + uint16_t cap; +}; + +static void links_mark_hist(void *ud, uint16_t id) +{ + struct links_mark *m = ud; + links_mark_id(id, m->mark, m->cap); +} + +static void links_mark_screen(const struct screen *s, struct links_mark *m) +{ + for (int y = 0; y < s->rows; y++) { + const struct srow *row = screen_get_row(s, y); + if (row->len == 0) + continue; + if (row->kind == ROW_CELLS) + links_mark_row(row->cells, row->len, m->mark, m->cap); + else + term_hist_text_mark_links(&row->text, links_mark_hist, m); + } +} + +static void links_gc(struct term *t, struct term_links *l) +{ + uint64_t mark[TERM_LINK_MAX / 64] = { 0 }; + struct links_mark m = { mark, l->capacity }; + links_mark_id(t->cur_link, mark, l->capacity); + links_mark_screen(&t->scr, &m); + if (t->alternate && t->alt_allocated) + links_mark_screen(&t->alt_scr, &m); + term_hist_mark_links(&t->hist, links_mark_hist, &m); + l->free_count = 0; + for (int i = 0; i + 1 < l->next; i++) { + if (l->uri[i] && !(mark[i >> 6] & (1ull << (i & 63)))) { + free(l->uri[i]); + l->uri[i] = NULL; + } + if (!l->uri[i]) + l->free_ids[l->free_count++] = (uint16_t)(i + 1); + } + if (l->hash_table && l->hash_cap) { + memset(l->hash_table, 0, sizeof(*l->hash_table) * l->hash_cap); + for (int i = 0; i + 1 < l->next; i++) { + if (l->uri[i]) + links_insert_hash(l, hash_uri(l->uri[i]), (uint16_t)(i + 1)); + } + } +} + +static uint16_t links_find(const struct term_links *l, const char *uri, uint64_t h) +{ + if (!l->hash_table || !l->hash_cap) + return 0; + uint32_t mask = l->hash_cap - 1; + uint32_t idx = (uint32_t)h & mask; + for (uint32_t probe = 0; probe <= mask; probe++) { + const struct link_hash_slot *slot = &l->hash_table[idx]; + if (slot->hash == 0) + return 0; + if (slot->hash == h) { + uint16_t id = slot->id; + if (id && id <= l->capacity && l->uri[id - 1] && !strcmp(l->uri[id - 1], uri)) + return id; + } + idx = (idx + 1) & mask; + } + return 0; +} + +static uint16_t links_alloc(struct term *t, struct term_links *l) +{ + if (l->free_count) + return l->free_ids[--l->free_count]; + if (l->next <= l->capacity || links_grow(l)) + return l->next++; + links_gc(t, l); + if (l->free_count) + return l->free_ids[--l->free_count]; + return 0; +} + +void term_osc8(struct term *t, const char *params) +{ + const char *sep = strchr(params, ';'); + if (!sep) + return; + const char *uri = sep + 1; + if (!*uri) { + t->cur_link = 0; + return; + } + if (strlen(uri) > TERM_LINK_URI_MAX) { + t->cur_link = 0; + return; + } + struct term_links *l = t->links; + if (!l) { + l = calloc(1, sizeof(*l)); + if (!l) { + t->cur_link = 0; + return; + } + l->next = 1; + t->links = l; + } + uint64_t h = hash_uri(uri); + uint16_t id = links_find(l, uri, h); + if (!id) { + id = links_alloc(t, l); + if (!id) { + t->cur_link = 0; + return; + } + l->uri[id - 1] = term_strdup(uri); + if (!l->uri[id - 1]) { + l->free_ids[l->free_count++] = id; + t->cur_link = 0; + return; + } + links_insert_hash(l, h, id); + } + t->cur_link = id; +} + +const char *term_osc8_url_at(const struct term *t, int col, int row) +{ + const struct term_links *l = t->links; + if (!l || col < 0 || col >= t->cols) + return NULL; + const struct cell *cells = term_screen_view_row(t, row); + if (!cells) + return NULL; + uint16_t id = cells[col].pad; + if (!id || id > l->capacity || + !(cells[col].flags & CELL_LINK)) + return NULL; + return l->uri[id - 1]; +} + +void term_osc8_cleanup(struct term *t) +{ + struct term_links *l = t->links; + if (!l) + return; + for (uint16_t i = 0; i < l->capacity; i++) + free(l->uri[i]); + free(l->uri); + free(l->free_ids); + free(l->hash_table); + free(l); + t->links = NULL; + t->cur_link = 0; +} diff --git a/src/osc/meson.build b/src/osc/meson.build new file mode 100644 index 0000000..a3b9efa --- /dev/null +++ b/src/osc/meson.build @@ -0,0 +1,6 @@ +term_osc_src = files( + 'clipboard.c', + 'dispatch.c', + 'link.c', + 'misc.c', +) diff --git a/src/osc/misc.c b/src/osc/misc.c new file mode 100644 index 0000000..5d6b506 --- /dev/null +++ b/src/osc/misc.c @@ -0,0 +1,69 @@ +#include "osc.h" +#include "../internal.h" +#include <stdlib.h> +#include <string.h> + +static void osc7_cwd(struct term *t, const char *uri) +{ + if (!term_has_prefix_ci(uri, "file://")) + return; + const char *p = uri + 7; + const char *slash = strchr(p, '/'); + const char *path = slash ? slash : p; + char *cwd = malloc(strlen(path) + 1); + if (!cwd) + return; + char *d = cwd; + while (*path) { + int hi, lo; + if (*path == '%' && (hi = term_hex_nib(path[1])) >= 0 && + (lo = term_hex_nib(path[2])) >= 0) { + *d++ = (char)(hi * 16 + lo); + path += 3; + } else { + *d++ = *path++; + } + } + *d = '\0'; + free(t->cwd); + t->cwd = cwd; +} + +static void osc777_notify(struct term *t, const char *params) +{ + if (strncmp(params, "notify;", 7)) + return; + const char *title = params + 7; + const char *sep = strchr(title, ';'); + if (!sep) + return; + char *wsep = (char *)sep; + *wsep = '\0'; + t->cb.notify(t->cb.ud, title, sep + 1); + *wsep = ';'; +} + +void term_osc_misc_dispatch(struct term *t, long code, const char *params) +{ + if (!params) + return; + switch (code) { + case 7: + osc7_cwd(t, params); + break; + case 9: + if (t->cb.notify && strncmp(params, "4;", 2)) + t->cb.notify(t->cb.ud, "wsap_term", params); + break; + case 22: + free(t->pointer_shape); + t->pointer_shape = *params ? term_strdup(params) : NULL; + break; + case 777: + if (t->cb.notify) + osc777_notify(t, params); + break; + default: + break; + } +} diff --git a/src/osc/osc.h b/src/osc/osc.h new file mode 100644 index 0000000..1398534 --- /dev/null +++ b/src/osc/osc.h @@ -0,0 +1,30 @@ +#ifndef TERM_OSC_H +#define TERM_OSC_H + +#include <stdbool.h> +#include <stddef.h> +#include <stdint.h> + +struct term; + +#define TERM_OSC_MAX (16u << 20) + +size_t term_osc_room(const struct term *t); +char *term_osc_tail(struct term *t); +void term_osc_commit(struct term *t, size_t n); +bool term_osc_grow(struct term *t); + +char *term_osc_data(struct term *t); + +void term_osc_finish(struct term *t); + +void term_osc_cleanup(struct term *t); + +void term_osc8(struct term *t, const char *params); +void term_osc52(struct term *t, const char *params); +void term_osc_misc_dispatch(struct term *t, long code, const char *params); + +const char *term_osc8_url_at(const struct term *t, int col, int row); +void term_osc8_cleanup(struct term *t); + +#endif diff --git a/src/palette.c b/src/palette.c new file mode 100644 index 0000000..1b27e3a --- /dev/null +++ b/src/palette.c @@ -0,0 +1,492 @@ +#include "palette.h" +#include "internal.h" + +#include <math.h> +#include <stdbool.h> +#include <stdint.h> +#include <string.h> + +/* Linear light for each byte value of an sRGB channel. The entries are the + * transfer curve, evaluated once and stored, so every host reads the same ones. */ +static const float srgb_linear[256] = { + 0x0p+0f, 0x1.3e4568p-12f, 0x1.3e4568p-11f, 0x1.dd681cp-11f, + 0x1.3e4568p-10f, 0x1.8dd6c2p-10f, 0x1.dd681cp-10f, 0x1.167cbap-9f, + 0x1.3e4568p-9f, 0x1.660e16p-9f, 0x1.8dd6c2p-9f, 0x1.b6a31ap-9f, + 0x1.e1e31ap-9f, 0x1.07c38cp-8f, 0x1.1fcc2cp-8f, 0x1.390ffap-8f, + 0x1.53936ep-8f, 0x1.6f5adep-8f, 0x1.8c6a92p-8f, 0x1.aac6c2p-8f, + 0x1.ca7382p-8f, 0x1.eb74ep-8f, 0x1.06e76ap-7f, 0x1.18c2a4p-7f, + 0x1.2b4e06p-7f, 0x1.3e8b7cp-7f, 0x1.527cd6p-7f, 0x1.6723eep-7f, + 0x1.7c8292p-7f, 0x1.929a86p-7f, 0x1.a96d8ep-7f, 0x1.c0fd62p-7f, + 0x1.d94bbep-7f, 0x1.f25a44p-7f, 0x1.06155p-6f, 0x1.135f3ep-6f, + 0x1.210bb6p-6f, 0x1.2f1b8ap-6f, 0x1.3d8f84p-6f, 0x1.4c6866p-6f, + 0x1.5ba6fap-6f, 0x1.6b4c02p-6f, 0x1.7b584p-6f, 0x1.8bcc72p-6f, + 0x1.9ca956p-6f, 0x1.adefaap-6f, 0x1.bfa02p-6f, 0x1.d1bb72p-6f, + 0x1.e44258p-6f, 0x1.f73582p-6f, 0x1.054ad2p-5f, 0x1.0f31b8p-5f, + 0x1.194fccp-5f, 0x1.23a55ep-5f, 0x1.2e32c6p-5f, 0x1.38f85cp-5f, + 0x1.43f678p-5f, 0x1.4f2d64p-5f, 0x1.5a9d76p-5f, 0x1.6647p-5f, + 0x1.722a56p-5f, 0x1.7e47c6p-5f, 0x1.8a9fa2p-5f, 0x1.973238p-5f, + 0x1.a3ffdep-5f, 0x1.b108d4p-5f, 0x1.be4d6ep-5f, 0x1.cbcdfcp-5f, + 0x1.d98ac8p-5f, 0x1.e7841ep-5f, 0x1.f5ba52p-5f, 0x1.0216cep-4f, + 0x1.096f2ap-4f, 0x1.10e66p-4f, 0x1.187c94p-4f, 0x1.2031ecp-4f, + 0x1.28068ap-4f, 0x1.2ffa94p-4f, 0x1.380e2cp-4f, 0x1.404176p-4f, + 0x1.489496p-4f, 0x1.5107aep-4f, 0x1.599aep-4f, 0x1.624e5p-4f, + 0x1.6b2224p-4f, 0x1.741676p-4f, 0x1.7d2b6ap-4f, 0x1.866124p-4f, + 0x1.8fb7c4p-4f, 0x1.992f6cp-4f, 0x1.a2c83ep-4f, 0x1.ac8258p-4f, + 0x1.b65ddep-4f, 0x1.c05aeep-4f, 0x1.ca79aap-4f, 0x1.d4ba32p-4f, + 0x1.df1ca4p-4f, 0x1.e9a122p-4f, 0x1.f447dp-4f, 0x1.ff10c2p-4f, + 0x1.04fe0ep-3f, 0x1.0a85p-3f, 0x1.101d46p-3f, 0x1.15c6fp-3f, + 0x1.1b820ap-3f, 0x1.214ea8p-3f, 0x1.272cd6p-3f, 0x1.2d1ca4p-3f, + 0x1.331e2p-3f, 0x1.39315ap-3f, 0x1.3f566p-3f, 0x1.458d46p-3f, + 0x1.4bd612p-3f, 0x1.5230d8p-3f, 0x1.589dap-3f, 0x1.5f1c82p-3f, + 0x1.65ad88p-3f, 0x1.6c50c2p-3f, 0x1.73063cp-3f, 0x1.79ce06p-3f, + 0x1.80a82cp-3f, 0x1.8794bep-3f, 0x1.8e93c8p-3f, 0x1.95a55cp-3f, + 0x1.9cc984p-3f, 0x1.a4004ep-3f, 0x1.ab49ccp-3f, 0x1.b2a606p-3f, + 0x1.ba1516p-3f, 0x1.c196f8p-3f, 0x1.c92bcp-3f, 0x1.d0d37ep-3f, + 0x1.d88e4p-3f, 0x1.e05c18p-3f, 0x1.e83d08p-3f, 0x1.f0312p-3f, + 0x1.f8387p-3f, 0x1.002984p-2f, 0x1.044078p-2f, 0x1.08611ap-2f, + 0x1.0c8b74p-2f, 0x1.10bf8ap-2f, 0x1.14fd64p-2f, 0x1.194506p-2f, + 0x1.1d967ap-2f, 0x1.21f1c2p-2f, 0x1.2656e8p-2f, 0x1.2ac5fp-2f, + 0x1.2f3ee2p-2f, 0x1.33c1c4p-2f, 0x1.384e9cp-2f, 0x1.3ce56ep-2f, + 0x1.418644p-2f, 0x1.463124p-2f, 0x1.4ae61p-2f, 0x1.4fa512p-2f, + 0x1.546e3p-2f, 0x1.59416ep-2f, 0x1.5e1ed2p-2f, 0x1.630666p-2f, + 0x1.67f83p-2f, 0x1.6cf43p-2f, 0x1.71fa6ep-2f, 0x1.770af2p-2f, + 0x1.7c25c2p-2f, 0x1.814ae2p-2f, 0x1.867a5ap-2f, 0x1.8bb42ep-2f, + 0x1.90f866p-2f, 0x1.964706p-2f, 0x1.9ba016p-2f, 0x1.a1039ap-2f, + 0x1.a67198p-2f, 0x1.abea16p-2f, 0x1.b16d1ap-2f, 0x1.b6faa8p-2f, + 0x1.bc92cap-2f, 0x1.c23582p-2f, 0x1.c7e2d6p-2f, 0x1.cd9acep-2f, + 0x1.d35d6cp-2f, 0x1.d92abap-2f, 0x1.df02bap-2f, 0x1.e4e574p-2f, + 0x1.ead2ecp-2f, 0x1.f0cb28p-2f, 0x1.f6ce2ep-2f, 0x1.fcdc04p-2f, + 0x1.017a5ap-1f, 0x1.048c1cp-1f, 0x1.07a34ep-1f, 0x1.0abff2p-1f, + 0x1.0de20cp-1f, 0x1.11099ep-1f, 0x1.1436acp-1f, 0x1.176936p-1f, + 0x1.1aa14p-1f, 0x1.1ddecep-1f, 0x1.2121e2p-1f, 0x1.246a7cp-1f, + 0x1.27b8a2p-1f, 0x1.2b0c56p-1f, 0x1.2e659ap-1f, 0x1.31c472p-1f, + 0x1.3528dep-1f, 0x1.3892e2p-1f, 0x1.3c0282p-1f, 0x1.3f77bep-1f, + 0x1.42f29cp-1f, 0x1.46731cp-1f, 0x1.49f94p-1f, 0x1.4d850cp-1f, + 0x1.511682p-1f, 0x1.54ada6p-1f, 0x1.584a7ap-1f, 0x1.5bedp-1f, + 0x1.5f953cp-1f, 0x1.63432ep-1f, 0x1.66f6d8p-1f, 0x1.6ab04p-1f, + 0x1.6e6f66p-1f, 0x1.72344cp-1f, 0x1.75fef8p-1f, 0x1.79cf6ap-1f, + 0x1.7da5a4p-1f, 0x1.8181a8p-1f, 0x1.85637cp-1f, 0x1.894b2p-1f, + 0x1.8d3896p-1f, 0x1.912be2p-1f, 0x1.952504p-1f, 0x1.992402p-1f, + 0x1.9d28dcp-1f, 0x1.a13396p-1f, 0x1.a5443p-1f, 0x1.a95aaep-1f, + 0x1.ad7714p-1f, 0x1.b1996p-1f, 0x1.b5c19ap-1f, 0x1.b9efcp-1f, + 0x1.be23d8p-1f, 0x1.c25ddcp-1f, 0x1.c69ddep-1f, 0x1.cae3d2p-1f, + 0x1.cf2fc6p-1f, 0x1.d381acp-1f, 0x1.d7d99ap-1f, 0x1.dc377ep-1f, + 0x1.e09b7p-1f, 0x1.e5055ep-1f, 0x1.e9755cp-1f, 0x1.edeb5cp-1f, + 0x1.f26772p-1f, 0x1.f6e98cp-1f, 0x1.fb71cp-1f, 0x1p+0f, +}; + +static inline float linf_to_srgbf(float c) +{ + return c > 0.0031308f ? 1.055f * powf(c, 1.0f / 2.4f) - 0.055f + : 12.92f * c; +} + +static const struct term_theme builtin_theme = { + .fg = 0xFFFFFFFFu, + .bg = 0xFF1E1E2Eu, + .cursor = 0xFF89B4FAu, + .ansi = { + 0xFF585B70, 0xFFCC6666, 0xFFB5BD68, 0xFFF0C674, + 0xFF81A2BE, 0xFFB294BB, 0xFF8ABEB7, 0xFFC5C8C6, + 0xFF666666, 0xFFD54E53, 0xFFB9CA4A, 0xFFE7C547, + 0xFF7AA6DA, 0xFFC397D8, 0xFF70C0B1, 0xFFEAEAEA, + }, +}; + +void term_theme_default(struct term_theme *theme) +{ + *theme = builtin_theme; +} + +static const struct term_theme high_contrast_theme = { + .fg = 0xFFFFFFFFu, + .bg = 0xFF000000u, + .cursor = 0xFF00FFFFu, + .ansi = { + 0xFF000000, 0xFFFF0000, 0xFF00FF00, 0xFFFFFF00, + 0xFF0000FF, 0xFFFF00FF, 0xFF00FFFF, 0xFFFFFFFF, + 0xFF808080, 0xFFFF5555, 0xFF55FF55, 0xFFFFFF55, + 0xFF5555FF, 0xFFFF55FF, 0xFF55FFFF, 0xFFFFFFFF, + }, +}; + +void term_theme_high_contrast(struct term_theme *theme) +{ + *theme = high_contrast_theme; +} + +/* Finds the slot for key and the default it resets to. Keys are + * foreground, background, and color0 to color15. */ +static bool palette_slot(struct term_theme *p, const struct term_theme *def, + const char *key, uint32_t **slot, + const uint32_t **fallback) +{ + if (!strcmp(key, "foreground")) { + *slot = &p->fg; + *fallback = &def->fg; + return true; + } + if (!strcmp(key, "background")) { + *slot = &p->bg; + *fallback = &def->bg; + return true; + } + if (strncmp(key, "color", 5)) + return false; + const char *digits = key + 5; + if (!*digits || (*digits == '0' && digits[1])) + return false; + int i = 0; + for (const char *d = digits; *d; d++) { + if (*d < '0' || *d > '9') + return false; + i = i * 10 + (*d - '0'); + if (i > 15) + return false; + } + *slot = &p->ansi[i]; + *fallback = &def->ansi[i]; + return true; +} + +bool term_theme_set_key(struct term_theme *p, const char *key, const char *val) +{ + struct term_theme def; + term_theme_default(&def); + uint32_t *slot; + const uint32_t *fallback; + if (!palette_slot(p, &def, key, &slot, &fallback)) + return false; + if (!val[0]) { + *slot = *fallback; + return true; + } + uint32_t v; + if (!term_color_parse(val, &v)) + return false; + *slot = v; + return true; +} + +static bool parse_hex_channel(const char *s, size_t len, uint8_t *out) +{ + if (len == 0 || len > 4) + return false; + unsigned v = 0; + for (size_t i = 0; i < len; i++) { + int d = term_hex_nib(s[i]); + if (d < 0) + return false; + v = (v << 4) | (unsigned)d; + } + static const unsigned max_for_len[5] = {0, 15, 255, 4095, 65535}; + unsigned max = max_for_len[len]; + *out = (uint8_t)((unsigned long)v * 255 / max); + return true; +} + +static bool parse_fraction(const char *s, size_t len, uint8_t *out) +{ + + if (len == 0) + return false; + bool seen_digit = false; + bool seen_dot = false; + unsigned integer = 0; + unsigned frac = 0; + unsigned frac_digits = 0; + size_t i = 0; + if (s[0] == '.') { + seen_dot = true; + i = 1; + } + for (; i < len; i++) { + char c = s[i]; + if (c >= '0' && c <= '9') { + seen_digit = true; + if (!seen_dot) { + integer = integer * 10 + (unsigned)(c - '0'); + if (integer > 1) + return false; + } else { + if (frac_digits < 9) { + frac = frac * 10 + (unsigned)(c - '0'); + frac_digits++; + } + } + } else if (c == '.' && !seen_dot) { + seen_dot = true; + } else { + return false; + } + } + if (!seen_digit) + return false; + double value = (double)integer; + if (frac_digits > 0) { + double f = (double)frac; + for (unsigned k = 0; k < frac_digits; k++) + f /= 10.0; + value += f; + } + if (value > 1.0) value = 1.0; + *out = (uint8_t)(value * 255.0 + 0.5); + return true; +} + +static const char *skip_leading_ws(const char *s) +{ + while (*s == ' ' || *s == '\t') + s++; + return s; +} + +static size_t rtrimmed_len(const char *s) +{ + size_t n = strlen(s); + while (n > 0 && (s[n - 1] == ' ' || s[n - 1] == '\t')) + n--; + return n; +} + +bool term_color_parse(const char *input, uint32_t *out) +{ + if (!input || !out) + return false; + const char *s = skip_leading_ws(input); + size_t len = rtrimmed_len(s); + if (len == 0) + return false; + + if (s[0] == '#') { + const char *body = s + 1; + size_t blen = len - 1; + size_t cw = blen / 3; + if (blen != 3 && blen != 6 && blen != 9 && blen != 12) + return false; + uint8_t r, g, b; + if (!parse_hex_channel(body, cw, &r) || + !parse_hex_channel(body + cw, cw, &g) || + !parse_hex_channel(body + 2 * cw, cw, &b)) + return false; + *out = term_rgb(r, g, b); + return true; + } + + if (len >= 4 && (s[0] == 'r' || s[0] == 'R') && + (s[1] == 'g' || s[1] == 'G') && (s[2] == 'b' || s[2] == 'B')) { + size_t i = 3; + bool intensity = false; + if ((s[i] == 'i' || s[i] == 'I')) { + intensity = true; + i++; + } + if (i >= len || s[i] != ':') + return false; + i++; + + const char *channels[3]; + size_t chlens[3]; + int ci = 0; + size_t start = i; + for (;; i++) { + if (i == len || s[i] == '/') { + if (ci >= 3) + return false; + channels[ci] = s + start; + chlens[ci] = i - start; + ci++; + if (i == len) + break; + start = i + 1; + } + } + if (ci != 3) + return false; + uint8_t r, g, b; + if (intensity) { + if (!parse_fraction(channels[0], chlens[0], &r) || + !parse_fraction(channels[1], chlens[1], &g) || + !parse_fraction(channels[2], chlens[2], &b)) + return false; + } else { + if (!parse_hex_channel(channels[0], chlens[0], &r) || + !parse_hex_channel(channels[1], chlens[1], &g) || + !parse_hex_channel(channels[2], chlens[2], &b)) + return false; + } + *out = term_rgb(r, g, b); + return true; + } + + if (len == 6 || len == 3) { + uint8_t r, g, b; + size_t cw = len / 3; + if (!parse_hex_channel(s, cw, &r) || + !parse_hex_channel(s + cw, cw, &g) || + !parse_hex_channel(s + 2 * cw, cw, &b)) + return false; + *out = term_rgb(r, g, b); + return true; + } + + return false; +} + +struct lab { float l, a, b; }; + +static struct lab lab_from_rgb(uint32_t argb) +{ + float r = srgb_linear[(argb >> 16) & 0xFF]; + float g = srgb_linear[(argb >> 8) & 0xFF]; + float b = srgb_linear[argb & 0xFF]; + + float x = (r * 0.4124564f + g * 0.3575761f + b * 0.1804375f) / 0.95047f; + float y = r * 0.2126729f + g * 0.7151522f + b * 0.0721750f; + float z = (r * 0.0193339f + g * 0.1191920f + b * 0.9503041f) / 1.08883f; + + x = x > 0.008856f ? cbrtf(x) : 7.787f * x + 16.0f / 116.0f; + y = y > 0.008856f ? cbrtf(y) : 7.787f * y + 16.0f / 116.0f; + z = z > 0.008856f ? cbrtf(z) : 7.787f * z + 16.0f / 116.0f; + + struct lab lab; + lab.l = 116.0f * y - 16.0f; + lab.a = 500.0f * (x - y); + lab.b = 200.0f * (y - z); + return lab; +} + +static uint32_t lab_to_rgb(struct lab lab) +{ + float y = (lab.l + 16.0f) / 116.0f; + float x = lab.a / 500.0f + y; + float z = y - lab.b / 200.0f; + + float x3 = x * x * x; + float y3 = y * y * y; + float z3 = z * z * z; + float xf = (x3 > 0.008856f ? x3 : (x - 16.0f / 116.0f) / 7.787f) * 0.95047f; + float yf = (y3 > 0.008856f ? y3 : (y - 16.0f / 116.0f) / 7.787f); + float zf = (z3 > 0.008856f ? z3 : (z - 16.0f / 116.0f) / 7.787f) * 1.08883f; + + float r = xf * 3.2404542f - yf * 1.5371385f - zf * 0.4985314f; + float g = -xf * 0.9692660f + yf * 1.8760108f + zf * 0.0415560f; + float b = xf * 0.0556434f - yf * 0.2040259f + zf * 1.0572252f; + + r = linf_to_srgbf(r); + g = linf_to_srgbf(g); + b = linf_to_srgbf(b); + + if (r < 0.0f) r = 0.0f; + if (r > 1.0f) r = 1.0f; + if (g < 0.0f) g = 0.0f; + if (g > 1.0f) g = 1.0f; + if (b < 0.0f) b = 0.0f; + if (b > 1.0f) b = 1.0f; + + uint8_t rb = (uint8_t)(r * 255.0f + 0.5f); + uint8_t gb = (uint8_t)(g * 255.0f + 0.5f); + uint8_t bb = (uint8_t)(b * 255.0f + 0.5f); + return term_rgb(rb, gb, bb); +} + +static struct lab lab_lerp(float t, struct lab a, struct lab b) +{ + struct lab r; + r.l = a.l + t * (b.l - a.l); + r.a = a.a + t * (b.a - a.a); + r.b = a.b + t * (b.b - a.b); + return r; +} + +/* + * Derive the 256-color palette from a theme. Entries 0-15 are the theme's + * ANSI colors. The 6x6x6 cube and the gray ramp interpolate in Lab space + * between the background, the ANSI colors, and the foreground, so they match + * the theme instead of the fixed xterm values. + */ +static void derive_palette(uint32_t *p, const struct term_theme *theme) +{ + struct lab base8[8]; + base8[0] = lab_from_rgb(theme->bg); + for (int i = 1; i < 7; i++) + base8[i] = lab_from_rgb(theme->ansi[i]); + base8[7] = lab_from_rgb(theme->fg); + + for (int i = 0; i < 16; i++) + p[i] = theme->ansi[i]; + p[TERM_PALETTE_FG] = theme->fg; + p[TERM_PALETTE_BG] = theme->bg; + p[TERM_PALETTE_CURSOR] = theme->cursor; + + bool is_light_theme = base8[7].l < base8[0].l; + if (is_light_theme) { + struct lab tmp = base8[0]; + base8[0] = base8[7]; + base8[7] = tmp; + } + + int idx = 16; + for (int ri = 0; ri < 6; ri++) { + float tr = (float)ri / 5.0f; + struct lab c0 = lab_lerp(tr, base8[0], base8[1]); + struct lab c1 = lab_lerp(tr, base8[2], base8[3]); + struct lab c2 = lab_lerp(tr, base8[4], base8[5]); + struct lab c3 = lab_lerp(tr, base8[6], base8[7]); + for (int gi = 0; gi < 6; gi++) { + float tg = (float)gi / 5.0f; + struct lab c4 = lab_lerp(tg, c0, c1); + struct lab c5 = lab_lerp(tg, c2, c3); + for (int bi = 0; bi < 6; bi++) + p[idx++] = lab_to_rgb(lab_lerp((float)bi / 5.0f, c4, c5)); + } + } + + for (int i = 0; i < 24; i++) { + float t = (float)(i + 1) / 25.0f; + p[idx++] = lab_to_rgb(lab_lerp(t, base8[0], base8[7])); + } +} + +static bool theme_is_zero(const struct term_theme *theme) +{ + static const struct term_theme zero; + return !memcmp(theme, &zero, sizeof(zero)); +} + +void term_palette_init(struct palette *p, const struct term_theme *theme) +{ + p->theme = theme && !theme_is_zero(theme) ? *theme : builtin_theme; + if (!p->theme.cursor) + p->theme.cursor = builtin_theme.cursor; + derive_palette(p->base, &p->theme); + memcpy(p->colors, p->base, sizeof(p->colors)); + p->version++; +} + +static bool valid_slot(int slot) +{ + return slot >= 0 && slot < TERM_PALETTE_SIZE; +} + +void term_palette_set(struct palette *p, int slot, uint32_t argb) +{ + if (!valid_slot(slot) || p->colors[slot] == argb) + return; + p->colors[slot] = argb; + p->version++; +} + +void term_palette_reset(struct palette *p, int slot) +{ + if (valid_slot(slot)) + term_palette_set(p, slot, p->base[slot]); +} + +void term_palette_reset_all(struct palette *p) +{ + if (memcmp(p->colors, p->base, sizeof(p->colors))) { + memcpy(p->colors, p->base, sizeof(p->colors)); + p->version++; + } +} diff --git a/src/palette.h b/src/palette.h new file mode 100644 index 0000000..5eef03e --- /dev/null +++ b/src/palette.h @@ -0,0 +1,34 @@ +#ifndef TERM_PALETTE_H +#define TERM_PALETTE_H + +#include <stdbool.h> +#include <stdint.h> + +#include "term.h" + +/* + * The colors of one terminal. base holds the colors that the theme gives; + * colors holds base plus the entries that the program changed with OSC 4, + * 10, 11, or 12. version changes whenever colors changes. + */ +struct palette { + uint32_t colors[TERM_PALETTE_SIZE]; + uint32_t base[TERM_PALETTE_SIZE]; + struct term_theme theme; + uint64_t version; +}; + +/* Set every entry from theme. An all-zero theme selects the built-in theme. */ +void term_palette_init(struct palette *p, const struct term_theme *theme); +/* Change one entry (0-255, TERM_PALETTE_FG, _BG, or _CURSOR). */ +void term_palette_set(struct palette *p, int slot, uint32_t argb); +/* Restore one entry, or all entries, to the theme color. */ +void term_palette_reset(struct palette *p, int slot); +void term_palette_reset_all(struct palette *p); + +static inline uint32_t palette_resolve(const struct palette *p, term_color c) +{ + return term_color_is_slot(c) ? p->colors[c & 0x1FFu] : c; +} + +#endif diff --git a/src/pen.h b/src/pen.h new file mode 100644 index 0000000..d7118b5 --- /dev/null +++ b/src/pen.h @@ -0,0 +1,165 @@ +#ifndef WSAP_TERM_PEN_H +#define WSAP_TERM_PEN_H + +#include "term.h" +#include <stdbool.h> +#include <stdint.h> + +struct term_pen { + uint32_t fg; + uint32_t bg; + uint32_t ul; + uint8_t underline; + bool bold : 1; + bool faint : 1; + bool italic : 1; + bool strike : 1; + bool overline : 1; + bool inverse : 1; +}; + +/* The pen of a stored cell. A cell has no underline color, so neither has + * this. */ +struct pen_attr { + uint32_t fg; + uint32_t bg; + uint16_t flags; + uint16_t link; +}; + +static inline struct pen_attr pen_attr_default(void) +{ + return (struct pen_attr){ .fg = TERM_COLOR_FG, .bg = TERM_COLOR_BG }; +} + +static inline bool pen_attr_is_default(const struct pen_attr *a) +{ + return a->fg == TERM_COLOR_FG && a->bg == TERM_COLOR_BG && a->flags == 0 && a->link == 0; +} + +static inline bool pen_attr_equal(const struct pen_attr *a, const struct pen_attr *b) +{ + return a->fg == b->fg && a->bg == b->bg && a->flags == b->flags && a->link == b->link; +} + +static inline uint32_t term_rgb(int r, int g, int b) +{ + return term_color_rgb((uint8_t)r, (uint8_t)g, (uint8_t)b); +} + +static inline void pen_reset(struct term_pen *p) +{ + p->fg = TERM_COLOR_FG; + p->bg = TERM_COLOR_BG; + p->ul = 0; + p->underline = 0; + p->bold = false; + p->faint = false; + p->italic = false; + p->strike = false; + p->overline = false; + p->inverse = false; +} + +static inline int sgr_extended_color(const int *params, int nparams, uint16_t subparams, int i, + uint32_t *color, bool *assigned) +{ + *assigned = false; + if (i + 1 >= nparams) + return 0; + if (i + 2 < nparams && params[i + 1] == 5) { + int index = params[i + 2]; + if (index >= 0 && index <= 255) { + *color = term_color_index(index); + *assigned = true; + } + return 2; + } + if (params[i + 1] != 2) + return 0; + int colon = 0; + while (i + 2 + colon < nparams && i + 2 + colon < 16 && + (subparams & (1u << (i + 2 + colon)))) + colon++; + int skip = colon >= 4 ? 1 : 0; + if (i + 4 + skip >= nparams) + return 0; + *color = term_rgb(params[i + 2 + skip], params[i + 3 + skip], params[i + 4 + skip]); + *assigned = true; + return 4 + skip; +} + +static inline bool pen_set_standard_color(struct term_pen *p, int code) +{ + if ((unsigned)(code - 30) < 8) + p->fg = term_color_index(code - 30); + else if ((unsigned)(code - 40) < 8) + p->bg = term_color_index(code - 40); + else if ((unsigned)(code - 90) < 8) + p->fg = term_color_index(code - 90 + 8); + else if ((unsigned)(code - 100) < 8) + p->bg = term_color_index(code - 100 + 8); + else + return false; + return true; +} + +static inline void sgr_apply(struct term_pen *pen, const int *params, int nparams, uint16_t subparams) +{ + if (nparams == 0) { + pen_reset(pen); + return; + } + for (int i = 0; i < nparams; i++) { + int p = params[i]; + switch (p) { + case 0: pen_reset(pen); break; + case 1: pen->bold = true; break; + case 2: pen->faint = true; break; + case 3: pen->italic = true; break; + case 4: { + int style = 1; + if (i + 1 < nparams && (subparams & (1u << (i + 1)))) + style = params[++i]; + pen->underline = (uint8_t)(style <= 5 ? style : 1); + break; + } + case 7: pen->inverse = true; break; + case 9: pen->strike = true; break; + case 21: pen->underline = 2; break; + case 22: pen->bold = false; pen->faint = false; break; + case 23: pen->italic = false; break; + case 24: pen->underline = 0; break; + case 27: pen->inverse = false; break; + case 29: pen->strike = false; break; + case 38: { + uint32_t c; bool ok; + i += sgr_extended_color(params, nparams, subparams, i, &c, &ok); + if (ok) pen->fg = c; + break; + } + case 39: pen->fg = TERM_COLOR_FG; break; + case 48: { + uint32_t c; bool ok; + i += sgr_extended_color(params, nparams, subparams, i, &c, &ok); + if (ok) pen->bg = c; + break; + } + case 49: pen->bg = TERM_COLOR_BG; break; + case 53: pen->overline = true; break; + case 55: pen->overline = false; break; + case 58: { + uint32_t c; bool ok; + i += sgr_extended_color(params, nparams, subparams, i, &c, &ok); + if (ok) pen->ul = c; + break; + } + case 59: pen->ul = 0; break; + default: + pen_set_standard_color(pen, p); + break; + } + } +} + +#endif diff --git a/src/scan.c b/src/scan.c new file mode 100644 index 0000000..497c95b --- /dev/null +++ b/src/scan.c @@ -0,0 +1,11 @@ +#include "compat.h" +#include "kernels.h" +#include "term.h" + +TERM_HOT +size_t term_scan_printable_span(const char *buf, size_t len, unsigned char hi) +{ + /* The only entry point that can run before the first terminal. */ + term_isa_init(); + return scan_printable_span(buf, len, hi); +} diff --git a/src/screen.c b/src/screen.c new file mode 100644 index 0000000..681c42b --- /dev/null +++ b/src/screen.c @@ -0,0 +1,1513 @@ +#include "screen.h" +#include "osc/osc.h" +#include <errno.h> +#include <limits.h> +#include <stdatomic.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> + +static void *grow(struct grow_buf *b, size_t bytes) +{ + if (bytes > b->cap) { + size_t cap = b->cap ? b->cap : 256; + while (cap < bytes) + cap *= 2; + void *data = realloc(b->data, cap); + if (!data) + return NULL; + b->data = data; + b->cap = cap; + } + return b->data; +} + +/* Make s an empty screen of cols x rows. It keeps its buffers when they are + * large enough, so a resize back and forth allocates nothing. */ +static bool screen_setup(struct screen *s, int cols, int rows, uint64_t *next_uid) +{ + int stride = (cols + 15) & ~15; + size_t text = TEXT_ROW_BYTES(stride), runs = TEXT_ROW_RUNS(stride); + size_t ascii = (size_t)rows * (size_t)cols; + size_t cells = (size_t)rows * (size_t)stride, bytes = ascii + (size_t)rows * (text + runs); + struct srow *slot = rows > s->slot_cap ? calloc((size_t)rows, sizeof(*slot)) : s->slot; + struct cell *cells_slab = cells > s->cells_cap ? calloc(cells, sizeof(*cells_slab)) : s->cells_slab; + uint8_t *text_slab = bytes > s->text_cap ? calloc(bytes, 1) : s->text_slab; + if (!slot || !cells_slab || !text_slab) { + if (slot != s->slot) free(slot); + if (cells_slab != s->cells_slab) free(cells_slab); + if (text_slab != s->text_slab) free(text_slab); + return false; + } + if (slot != s->slot) { + free(s->slot); + s->slot = slot; + s->slot_cap = rows; + } + if (cells_slab != s->cells_slab) { + free(s->cells_slab); + s->cells_slab = cells_slab; + s->cells_cap = cells; + } + if (text_slab != s->text_slab) { + free(s->text_slab); + s->text_slab = text_slab; + s->text_cap = bytes; + } + fill_cells(s->cells_slab, (int)cells, TERM_COLOR_FG, TERM_COLOR_BG); + s->rows = rows; + s->cols = cols; + s->top = 0; + s->stride = stride; + s->top_id = 0; + s->top_seg = 0; + for (int i = 0; i < rows; i++) { + struct srow *r = &s->slot[i]; + *r = (struct srow){ + .cells = s->cells_slab + (size_t)i * stride, + .ascii = s->text_slab + (size_t)i * cols, + .text = { + .text = s->text_slab + ascii + (size_t)i * (text + runs), + .storage = s->text_slab + ascii + (size_t)i * (text + runs), + .runs = { .data = s->text_slab + ascii + (size_t)i * (text + runs) + text, + .cap = (uint32_t)runs }, + }, + .uid = ++(*next_uid), + .version = 1, + .kind = ROW_CELLS, + .flags = RF_TAIL_PADDED, + }; + } + return true; +} + +static void screen_push_row_inner(struct term *t, struct srow *row) +{ + srow_materialize_text(row); + bool wrapped = (row->flags & RF_WRAPPED) != 0; + int cols = t->scr.cols; + if (row->kind == ROW_TEXT && !(wrapped && row->len < cols)) { + struct text_row *tr = &row->text; + /* Fill blanks are cells of the row, as a cells row stores them. */ + int fill = (tr->fill_to < cols ? tr->fill_to : cols) - tr->cells; + if (fill > 0) { + text_row_unshare(tr); + memset(tr->text + tr->text_len, ' ', (size_t)fill); + text_commit(row, fill, fill, fill, HLINE_NARROW, tr->fill.fg, tr->fill.bg, 0); + } + term_hist_push_text(&t->hist, tr, wrapped); + return; + } + /* The rest of a wrapped row is part of the line: blanks up to the edge. */ + if (TERM_EXPECT(wrapped && row->len < cols, 0)) { + srow_settle(row, cols); + if (!(row->flags & RF_TAIL_PADDED)) + fill_cells(row->cells + row->len, cols - row->len, TERM_COLOR_FG, TERM_COLOR_BG); + row->flags |= RF_TAIL_PADDED; + row->len = (uint16_t)cols; + } + term_hist_push_row(&t->hist, row->cells, row->len, wrapped, t->gstore); +} + +void term_screen_push_row(struct term *t, struct srow *row) +{ + screen_push_row_inner(t, row); + /* The push can drop more rows than it adds, so the view learns the new + * history size here and not before. */ + if (TERM_EXPECT(t->view.anchor.active, 0)) + term_hist_view_note_push(&t->view, &t->hist, t->cols); +} + +static void screen_free(struct screen *s) +{ + if (!s) + return; + free(s->slot); + free(s->cells_slab); + free(s->text_slab); + *s = (struct screen){ 0 }; +} + +/* Read the history count without changing its rows. */ +int term_screen_history_rows(const struct term *t) +{ + if (!t || !screen_has_history(t)) + return 0; + uint64_t rows = term_hist_view_rows(&t->view, &t->hist, t->cols); + return rows > INT_MAX ? INT_MAX : (int)rows; +} + +int term_history_rows(const struct term *t) +{ + if (!t) + return 0; + term_write_begin((struct term *)t); + int rows = term_screen_history_rows(t); + term_write_end((struct term *)t); + return rows; +} + +const struct cell *term_ring_row(const struct term *t, int row) +{ + if (!t) + return NULL; + int total = term_screen_history_rows(t); + if (row < 0 || row >= total + t->rows) + return NULL; + + if (row < total) { + uint64_t id; + int seg; + if (!term_hist_view_find_row(&t->view, &t->hist, row, t->cols, &id, &seg)) + return NULL; + struct cell *out = t->view_row_scratch; + int got = term_hist_decode_segment(&t->hist, id, seg, t->cols, out, t->gstore); + if (got < t->cols) { + fill_cells(out + got, t->cols - got, TERM_COLOR_FG, TERM_COLOR_BG); + int line_rows = term_hist_line_rows(&t->hist, id, t->cols); + if (seg + 1 < line_rows || (row + 1 == total && hist_is_open(&t->hist))) { + for (int x = got; x < t->cols; x++) + out[x].flags |= CELL_WRAP_PAD; + } + } + return out; + } + + int y = row - total; + const struct screen *s = term_active_screen(t); + struct srow *srow = screen_row_at(s, y); + if (!srow) + return NULL; + + struct cell *out = t->view_row_scratch; + if (srow->kind == ROW_CELLS) { + if (srow->len >= t->cols || (srow->flags & RF_TAIL_PADDED)) + return srow->cells; + if (srow->len > 0) + memcpy(out, srow->cells, (size_t)srow->len * sizeof(struct cell)); + fill_cells(out + srow->len, t->cols - srow->len, TERM_COLOR_FG, TERM_COLOR_BG); + return out; + } + + text_row_cells(srow, out, t->cols); + return out; +} + +const struct cell *term_screen_view_row(const struct term *t, int y) +{ + if (!t || y < 0 || y >= t->rows) + return NULL; + uint64_t id; + int seg, scr_y; + if (term_hist_view_row_pos(&t->view, &t->hist, t->cols, t->rows, screen_top_line(t), y, + &id, &seg, &scr_y)) { + struct cell *out = t->view_row_scratch; + int got = term_hist_decode_segment(&t->hist, id, seg, t->cols, out, t->gstore); + if (got < t->cols) { + fill_cells(out + got, t->cols - got, TERM_COLOR_FG, TERM_COLOR_BG); + int line_rows = term_hist_line_rows(&t->hist, id, t->cols); + if (seg + 1 < line_rows || (id + 1 == hist_end_id(&t->hist) && hist_is_open(&t->hist))) { + for (int x = got; x < t->cols; x++) + out[x].flags |= CELL_WRAP_PAD; + } + } + return out; + } + const struct screen *s = term_active_screen(t); + struct srow *srow = screen_row_at(s, scr_y); + if (!srow) + return NULL; + struct cell *out = t->view_row_scratch; + if (srow->kind == ROW_CELLS) { + if (srow->len >= t->cols || (srow->flags & RF_TAIL_PADDED)) + return srow->cells; + if (srow->len > 0) + memcpy(out, srow->cells, (size_t)srow->len * sizeof(struct cell)); + fill_cells(out + srow->len, t->cols - srow->len, TERM_COLOR_FG, TERM_COLOR_BG); + return out; + } + text_row_cells(srow, out, t->cols); + return out; +} + +static inline uint64_t mix64(uint64_t x) +{ + x ^= x >> 33; + x *= 0xff51afd7ed558ccdull; + x ^= x >> 33; + x *= 0xc4ceb9fe1a85ec53ull; + return x ^ (x >> 33); +} + +uint64_t term_screen_view_row_key(const struct term *t, int y) +{ + if (!t || y < 0 || y >= t->rows) + return 0; + uint64_t id; + int seg, scr_y; + if (term_hist_view_row_pos(&t->view, &t->hist, t->cols, t->rows, screen_top_line(t), y, + &id, &seg, &scr_y)) { + uint64_t line_ver = (id + 1 == hist_end_id(&t->hist) && hist_is_open(&t->hist)) + ? (term_hist_line_cells(&t->hist, id) << 32) : 0; + if (id == t->hist.base_cell_id) + line_ver ^= mix64(t->hist.base_cell); + return mix64(((uint64_t)id << 16) ^ (uint64_t)seg ^ line_ver ^ 0x517cc1b727220a95ull); + } + const struct screen *s = term_active_screen(t); + const struct srow *row = screen_row_at(s, scr_y); + if (!row) + return 0; + return mix64(row->uid ^ ((uint64_t)row->version << 32)); +} + +int term_view_offset(const struct term *t) +{ + if (!t) + return 0; + return term_hist_view_scroll_offset(&t->view, &t->hist, t->cols); +} + +void term_scroll_to(struct term *t, int offset) +{ + if (!t) + return; + term_write_begin(t); + if (t->alternate || offset < 0) + offset = 0; + struct view_anchor old = t->view.anchor; + term_hist_view_scroll_to(&t->view, &t->hist, t->cols, screen_top_line(t), offset); + if (old.active != t->view.anchor.active || old.line_id != t->view.anchor.line_id || + old.seg != t->view.anchor.seg) + t->full_gen++; + term_write_end(t); +} + +void term_scroll(struct term *t, int lines) +{ + if (!t) + return; + /* Read the alternate flag inside the writer section. A read before + * term_write_begin races with a feed on another thread. */ + term_write_begin(t); + if (t->alternate) { + term_write_end(t); + return; + } + struct view_anchor old = t->view.anchor; + term_hist_view_scroll(&t->view, &t->hist, t->cols, screen_top_line(t), lines); + if (old.active != t->view.anchor.active || old.line_id != t->view.anchor.line_id || + old.seg != t->view.anchor.seg) + t->full_gen++; + term_write_end(t); +} + +struct term_scrollback term_scrollback(const struct term *t) +{ + if (!t) + return (struct term_scrollback){ 0 }; + term_write_begin((struct term *)t); + struct term_scrollback sb = { 0 }; + if (!t->alternate) { + sb.offset = term_view_offset(t); + sb.total = term_screen_history_rows(t); + } + term_write_end((struct term *)t); + return sb; +} + +void term_screen_erase_line(struct term *t, int mode) +{ + t->pending_wrap = false; + int p = phys_row(t, t->cursor_y); + if (mode == 0) { + erase_span(t, p, t->cursor_x, erase_width(t, p)); + } else if (mode == 1) { + erase_span(t, p, 0, t->cursor_x + 1); + } else if (mode == 2) { + erase_span(t, p, 0, erase_width(t, p)); + } +} + +void term_screen_delete_chars(struct term *t, int n) +{ + if (t->cursor_x >= t->cols) + return; + t->pending_wrap = false; + int x = t->cursor_x; + int cols = t->cols; + if (n > cols - x) + n = cols - x; + int p = phys_row(t, t->cursor_y); + struct cell *row = row_cells(t, p); + struct srow *srow = term_row(t, t->cursor_y); + int len = srow->len; + clear_wide_edges(row, x, x + n, len); + if (len > cols) { + clear_cells_default(row + cols, len - cols); + len = cols; + } + if (len > x + n) { + memmove(&row[x], &row[x + n], (size_t)(len - (x + n)) * sizeof(struct cell)); + clear_cells_default(&row[len - n], n); + srow->len = (uint16_t)(len - n); + } else if (len > x) { + clear_cells_default(&row[x], len - x); + srow->len = (uint16_t)x; + } + srow->flags &= (uint8_t)~RF_TAIL_PADDED; + bool pristine = using_default_colors(t); + if (!pristine) { + int from = cols - n; + if (from < x) + from = x; + term_row_fill_gap(srow, from); + fill_cells(&row[from], cols - from, t->cur_fg, t->cur_bg); + srow->len = (uint16_t)cols; + } + row_set_wrapped(srow, false); + repair_wide_row(row, x - 1, len, cols, srow->len); + srow->version++; +} + +void term_screen_insert_chars(struct term *t, int n) +{ + if (t->cursor_x >= t->cols) + return; + t->pending_wrap = false; + int x = t->cursor_x; + int cols = t->cols; + if (n > cols - x) + n = cols - x; + int p = phys_row(t, t->cursor_y); + struct cell *row = row_cells(t, p); + struct srow *srow = term_row(t, t->cursor_y); + term_row_fill_gap(srow, x); + int len = srow->len; + if (x > 0 && x < len && (row[x].flags & CELL_TRAIL)) { + fill_cells(row + x - 1, 2, t->cur_fg, t->cur_bg); + srow->version++; + } + if (len > x) { + int shift = cols - (x + n); + if (shift > len - x) + shift = len - x; + if (shift > 0) + memmove(&row[x + n], &row[x], (size_t)shift * sizeof(struct cell)); + } + fill_cells(&row[x], n, t->cur_fg, t->cur_bg); + bool pristine = using_default_colors(t); + int new_len = len + n > cols ? cols : len + n; + if (pristine && len <= x) + new_len = len; + else if (new_len < x + n) + new_len = x + n; + if (new_len < cols) + clear_cells_default(row + new_len, cols - new_len); + srow->len = (uint16_t)new_len; + srow->flags &= (uint8_t)~RF_TAIL_PADDED; + row_set_wrapped(srow, false); + int repair_end = x + n + 1 > len + 1 ? x + n + 1 : len + 1; + repair_wide_row(row, x - 1, repair_end, cols, srow->len); + srow->version++; +} + +void term_screen_erase_all(struct term *t) +{ + struct screen *s = &t->scr; + bool pristine = using_default_colors(t); + for (int y = 0; y < s->rows; y++) { + int p = phys_row(t, y); + if (pristine && s->slot[p].len == 0 && !(s->slot[p].flags & RF_WRAPPED)) + continue; + erase_row(t, p); + } + t->full_gen++; +} + +#if defined(TERM_KERNEL_AVX2) +/* The inner columns [1, len - 1): the callers probed both ends. */ +TERM_TARGET_AVX2 +static bool text_has_text_avx2(const uint8_t *text, int len) +{ + int x = 1; + const __m256i vspace = _mm256_set1_epi8(' '); + for (; x + 32 <= len - 1; x += 32) { + __m256i chunk = _mm256_loadu_si256((const __m256i *)(text + x)); + __m256i diff = _mm256_subs_epu8(chunk, vspace); + if (!_mm256_testz_si256(diff, diff)) + return true; + } + for (; x < len - 1; x++) + if (text[x] > ' ') + return true; + return false; +} + +TERM_TARGET_AVX2 +static bool cells_has_text_avx2(const struct cell *cells, int len) +{ + int x = 1; + const __m256i vspace = _mm256_set1_epi32(0x20); + const __m256i mask = _mm256_setr_epi32(-1, 0, 0, 0, -1, 0, 0, 0); + for (; x + 4 <= len - 1; x += 4) { + __m256i v0 = _mm256_loadu_si256((const __m256i *)(cells + x)); + __m256i v1 = _mm256_loadu_si256((const __m256i *)(cells + x + 2)); + __m256i cmp0 = _mm256_cmpgt_epi32(v0, vspace); + __m256i cmp1 = _mm256_cmpgt_epi32(v1, vspace); + __m256i hits = _mm256_and_si256(_mm256_or_si256(cmp0, cmp1), mask); + if (!_mm256_testz_si256(hits, hits)) + return true; + } + for (; x < len - 1; x++) + if (cells[x].c > ' ') + return true; + return false; +} +#endif + +static inline bool text_has_text(const uint8_t *text, int len) +{ + if (len <= 0) + return false; + /* Probe column 0 first. Most terminal lines start with visible text. */ + if (text[0] > ' ') + return true; + if (len == 1) + return false; + /* Probe the line tail next. Indented lines often end with visible text. */ + if (text[len - 1] > ' ') + return true; + + int x = 1; +#if defined(__aarch64__) + const uint8x16_t vspace = vdupq_n_u8(' '); + for (; x + 16 <= len - 1; x += 16) { + uint8x16_t chunk = vld1q_u8(text + x); + uint8x16_t cmp = vcgtq_u8(chunk, vspace); + if (vmaxvq_u8(cmp) != 0) + return true; + } +#elif defined(__x86_64__) +#if defined(TERM_KERNEL_AVX2) + if (term_use_avx2()) + return text_has_text_avx2(text, len); +#endif + const __m128i vspace = _mm_set1_epi8(' '); + for (; x + 16 <= len - 1; x += 16) { + __m128i chunk = _mm_loadu_si128((const __m128i *)(text + x)); + __m128i diff = _mm_subs_epu8(chunk, vspace); + if (_mm_movemask_epi8(_mm_cmpeq_epi8(diff, _mm_setzero_si128())) != 0xFFFF) + return true; + } +#endif + for (; x < len - 1; x++) + if (text[x] > ' ') + return true; + return false; +} + +static inline bool cells_has_text(const struct cell *cells, int len) +{ + if (len <= 0) + return false; + /* Probe column 0 first. */ + if (cells[0].c > ' ') + return true; + if (len == 1) + return false; + /* Probe the line tail next. */ + if (cells[len - 1].c > ' ') + return true; + + int x = 1; +#if defined(__aarch64__) + const uint32x4_t vspace = vdupq_n_u32(' '); + for (; x + 4 <= len - 1; x += 4) { + uint32x4x4_t v = vld4q_u32((const uint32_t *)(cells + x)); + uint32x4_t cmp = vcgtq_u32(v.val[0], vspace); + if (vmaxvq_u32(cmp) != 0) + return true; + } +#elif defined(TERM_KERNEL_AVX2) + if (term_use_avx2()) + return cells_has_text_avx2(cells, len); +#endif + for (; x < len - 1; x++) + if (cells[x].c > ' ') + return true; + return false; +} + +static inline bool srow_has_text(const struct srow *r, int cols) +{ + int len = r->len < cols ? r->len : cols; + if (len <= 0) + return false; + if (r->kind == ROW_TEXT) + return text_has_text(r->text.text, r->text.text_len < len ? r->text.text_len : len); + return cells_has_text(r->cells, len); +} + +void term_screen_erase(struct term *t, int mode) +{ + if (mode == 3) { + term_hist_clear(&t->hist); + term_hist_view_clear(&t->view); + term_select_clear(t); + t->full_gen++; + return; + } + if (mode == 2) { + int last_active = -1; + bool tail_clean = using_default_colors(t); + int rows = t->rows; + int cols = t->cols; + const struct srow *slots = t->scr.slot; + int p = t->scr.top ? t->scr.top - 1 : rows - 1; + for (int y = rows - 1; y >= 0; y--) { + const struct srow *r = &slots[p]; + p = p ? p - 1 : rows - 1; + if (TERM_EXPECT(r->len == 0, 1)) { + if (TERM_EXPECT((r->flags & RF_WRAPPED) != 0, 0)) + tail_clean = false; + continue; + } + if (srow_has_text(r, cols)) { + last_active = y; + break; + } + tail_clean = false; + } + if (last_active >= 0 && !t->alternate) { + for (int y = 0; y <= last_active; y++) + term_screen_scroll_up(t); + if (tail_clean) { + term_hist_view_reset_bottom(&t->view); + return; + } + } + term_screen_erase_all(t); + term_hist_view_reset_bottom(&t->view); + t->full_gen++; + } else if (mode == 0) { + int cur_p = phys_row(t, t->cursor_y); + erase_span(t, cur_p, t->cursor_x, erase_width(t, cur_p)); + bool pristine = using_default_colors(t); + for (int r = t->cursor_y + 1; r < t->rows; r++) { + int p = phys_row(t, r); + if (pristine && t->scr.slot[p].len == 0 && !(t->scr.slot[p].flags & RF_WRAPPED)) + continue; + erase_row(t, p); + } + t->scr.slot[cur_p].version++; + } else if (mode == 1) { + bool pristine = using_default_colors(t); + for (int r = 0; r < t->cursor_y; r++) { + int p = phys_row(t, r); + if (pristine && t->scr.slot[p].len == 0 && !(t->scr.slot[p].flags & RF_WRAPPED)) + continue; + erase_row(t, p); + } + int cur_p = phys_row(t, t->cursor_y); + erase_span(t, cur_p, 0, t->cursor_x + 1); + t->scr.slot[cur_p].version++; + } +} + +void term_screen_scroll_region(struct term *t, int top, int bottom, int count, + bool down, bool history) +{ + if (count <= 0 || top > bottom || top < 0 || bottom >= t->rows) + return; + int height = bottom - top + 1; + if (count > height) + count = height; + + if (!down && history && top == 0 && bottom == t->rows - 1) { + while (count-- > 0) + term_screen_scroll_up(t); + return; + } + + if (!down && history && top == 0 && !t->alternate) { + for (int i = 0; i < count; i++) { + term_screen_scroll_up(t); + if (bottom + 1 < t->rows) + term_screen_scroll_region(t, bottom, t->rows - 1, 1, true, false); + } + return; + } + + struct screen *s = term_active_screen(t); + bool def = using_default_colors(t); + + if (count == height) { + for (int y = top; y <= bottom; y++) { + struct srow *dst = screen_get_row(s, y); + dst->kind = ROW_CELLS; + dst->uid = ++t->next_row_uid; + dst->version++; + if (def) { + dst->len = 0; + dst->flags = 0; + } else { + fill_cells(dst->cells, t->cols, t->cur_fg, t->cur_bg); + dst->len = (uint16_t)t->cols; + dst->flags = RF_TAIL_PADDED; + } + } + } else { + struct srow tmp[256]; + struct srow *buf = (height <= 256) ? tmp : malloc((size_t)height * sizeof(struct srow)); + if (!buf) + return; + + for (int i = 0; i < height; i++) + buf[i] = *screen_get_row(s, top + i); + + for (int i = 0; i < height; i++) { + int src_i = down ? i - count : i + count; + struct srow *dst = screen_get_row(s, top + i); + if (src_i >= 0 && src_i < height) { + *dst = buf[src_i]; + dst->version++; + } else { + /* The row takes the buffers of a row that left the region. */ + int dead_i = down ? (height - count + i) : (i - (height - count)); + dst->cells = buf[dead_i].cells; + dst->ascii = buf[dead_i].ascii; + dst->text = buf[dead_i].text; + dst->kind = ROW_CELLS; + dst->uid = ++t->next_row_uid; + dst->version = 1; + if (def) { + dst->len = 0; + dst->flags = 0; + } else { + fill_cells(dst->cells, t->cols, t->cur_fg, t->cur_bg); + dst->len = (uint16_t)t->cols; + dst->flags = RF_TAIL_PADDED; + } + } + } + if (buf != tmp) + free(buf); + } + + if (!t->alternate && t->prompt_y >= top && t->prompt_y <= bottom) { + t->prompt_y += down ? count : -count; + if (t->prompt_y < top || t->prompt_y > bottom) { + t->prompt_y = -1; + t->has_prompt = false; + } + } + t->full_gen++; +} + +static bool term_resize_internal(struct term *t, int cols, int rows); + +void term_screen_set_alternate(struct term *t, bool enabled) +{ + if (t->alternate == enabled) + return; + if (enabled) { + if (!t->alt_allocated || t->alt_scr.cols != t->cols || t->alt_scr.rows != t->rows) { + if (!screen_setup(&t->alt_scr, t->cols, t->rows, &t->next_row_uid)) + return; + t->alt_allocated = true; + } else { + for (int y = 0; y < t->alt_scr.rows; y++) { + struct srow *r = &t->alt_scr.slot[y]; + r->kind = ROW_CELLS; + r->len = 0; + r->flags = 0; + r->uid = ++t->next_row_uid; + r->version++; + } + t->alt_scr.top = 0; + } + struct screen tmp_scr = t->scr; + t->scr = t->alt_scr; + t->alt_scr = tmp_scr; + t->normal_saved_cursor = term_cursor_save(t); + t->normal_saved_decsc = t->saved; + t->normal_cursor_visible = t->cursor_visible; + t->normal_cursor_style = (uint8_t)t->cursor_style; + t->normal_cursor_blink = t->cursor_blink; + t->normal_cols = t->cols; + t->normal_rows = t->rows; + t->alternate = true; + t->cursor_x = 0; + t->cursor_y = 0; + t->pending_wrap = false; + t->scroll_top = 0; + t->scroll_bottom = t->rows - 1; + t->left_margin = 0; + t->right_margin = t->cols - 1; + /* The alternate screen has no history to show. */ + term_hist_view_reset_bottom(&t->view); + term_select_clear(t); + t->full_gen++; + } else { + int cols = t->cols; + int rows = t->rows; + struct screen tmp_scr = t->scr; + t->scr = t->alt_scr; + t->alt_scr = tmp_scr; + t->cols = t->scr.cols; + t->rows = t->scr.rows; + t->alternate = false; + t->saved = t->normal_saved_decsc; + t->cursor_visible = t->normal_cursor_visible; + t->cursor_style = (enum term_cursor_style)t->normal_cursor_style; + t->cursor_blink = t->normal_cursor_blink; + term_cursor_apply(t, &t->normal_saved_cursor); + t->cursor_x = t->normal_saved_cursor.x; + t->cursor_y = t->normal_saved_cursor.y; + t->pending_wrap = t->normal_saved_cursor.pending_wrap; + if (cols != t->cols || rows != t->rows) + term_resize_internal(t, cols, rows); + t->scroll_top = 0; + t->scroll_bottom = t->rows - 1; + t->left_margin = 0; + t->right_margin = t->cols - 1; + /* The alternate screen has no history to show. */ + term_hist_view_reset_bottom(&t->view); + term_select_clear(t); + t->full_gen++; + } +} + +static int scrollback_lines(int requested) +{ + if (requested == 0) + return TERM_SCROLLBACK_DEFAULT; + if (requested < 0) + return 0; + return requested > 1000000 ? 1000000 : requested; +} + +static atomic_int tables_state; + +/* Build the shared lookup tables once. A second caller waits until the first + * is done. */ +static void tables_init(void) +{ + if (atomic_load_explicit(&tables_state, memory_order_acquire) == 2) + return; + int expected = 0; + if (atomic_compare_exchange_strong(&tables_state, &expected, 1)) { + term_isa_init(); + term_text_tables_init(); + term_snapshot_tables_init(); + atomic_store_explicit(&tables_state, 2, memory_order_release); + return; + } + while (atomic_load_explicit(&tables_state, memory_order_acquire) != 2) + term_cpu_relax(); +} + +bool term_tables_ready(void) +{ + return atomic_load_explicit(&tables_state, memory_order_relaxed) == 2; +} + +struct term *term_new(const struct term_options *opt) +{ + tables_init(); + static const struct term_options defaults; + if (!opt) + opt = &defaults; + int cols = opt->size.x < 1 ? 80 : opt->size.x; + int rows = opt->size.y < 1 ? 24 : opt->size.y; + if (cols > TERM_MAX_COLS) + cols = TERM_MAX_COLS; + + static _Atomic uint64_t next_serial = 1; + struct term *t = calloc(1, sizeof(*t)); + if (!t) + return NULL; + atomic_init(&t->seq, 0); + atomic_init(&t->writer_tid, 0); + t->write_depth = 0; + t->serial = atomic_fetch_add_explicit(&next_serial, 1, memory_order_relaxed); + t->cols = cols; + t->rows = rows; + t->scroll_bottom = rows - 1; + t->right_margin = cols - 1; + t->prompt_y = -1; + t->sb_capacity = scrollback_lines(opt->scrollback); + t->cb = opt->callbacks; + term_palette_init(&t->palette, &opt->theme); + + t->next_row_uid = 1; + if (!screen_setup(&t->scr, cols, rows, &t->next_row_uid)) { + free(t); + return NULL; + } + if (!term_hist_init(&t->hist, t->sb_capacity, 0)) { + screen_free(&t->scr); + free(t); + return NULL; + } + t->hist.wrap_cols = (uint16_t)cols; + term_hist_view_init(&t->view); + + t->gstore = term_gstore_new(t); + t->view_scratch_cap = cols > 256 ? cols : 256; + t->view_row_scratch = calloc((size_t)t->view_scratch_cap, sizeof(struct cell)); + t->tabs_cap = cols > 256 ? cols : 256; + t->tabs = calloc((size_t)t->tabs_cap, 1); + + if (!t->gstore || !t->view_row_scratch || !t->tabs) { + term_free(t); + return NULL; + } + + t->cur_fg = TERM_COLOR_FG; + t->cur_bg = TERM_COLOR_BG; + t->cursor_visible = true; + t->default_cursor_style = opt->cursor_style > TERM_CURSOR_BAR ? TERM_CURSOR_BLOCK : opt->cursor_style; + t->default_cursor_blink = !opt->cursor_steady; + t->cursor_style = t->default_cursor_style; + t->cursor_blink = t->default_cursor_blink; + t->autowrap = true; + t->charsets[0] = t->charsets[1] = 'B'; + t->saved = term_saved_cursor_initial(); + memcpy(t->tcap_name, "wsap_term", sizeof("wsap_term")); + term_tabs_default(t); + t->full_gen = 1; + for (int j = 0; j < TERM_WIDTH_SLOTS; j++) + t->text_width.lo[j] = 0xFFFFFFFFu; + + return t; +} + +void term_free(struct term *t) +{ + if (!t) + return; + screen_free(&t->scr); + screen_free(&t->alt_scr); + screen_free(&t->spare); + struct grow_buf *bufs[] = { &t->resize.row, &t->resize.rows, &t->resize.cells, &t->resize.line, + &t->resize.pull, &t->resize.rest, &t->resize.lens }; + for (size_t i = 0; i < sizeof(bufs) / sizeof(bufs[0]); i++) + free(bufs[i]->data); + free(t->ascii_ends.data); + free(t->ascii_tail.data); + term_hist_free(&t->hist); + term_hist_view_free(&t->view); + free(t->view_row_scratch); + free(t->tabs); + term_osc_cleanup(t); + term_gstore_free(t->gstore); + free(t); +} + +/* ---- Resize & Reflow ---- */ + +struct reflow_target { + int cols, rows; + struct cell *cells; + int x; + int count; + int cursor_x, cursor_y, saved_x, saved_y, prompt_y; + bool cursor_set, saved_set, prompt_set, pending_wrap; + + struct reflow_out { + int len; + bool wrapped; + } *out; + struct cell *out_cells; + int out_cap; + struct grow_buf *out_buf, *cells_buf; /* they keep their size across resizes */ +}; + +static inline struct cell *reflow_out_row_cells(const struct reflow_target *r, int i) +{ + return r->out_cells + (size_t)i * (size_t)r->cols; +} + +/* Returns false when a row buffer cannot grow. The caller then stops the + * reflow before the terminal changes. */ +static bool reflow_target_end_row(struct reflow_target *r, bool wrapped) +{ + if (r->count == r->out_cap) { + int cap = r->out_cap ? r->out_cap * 2 : (r->rows + 16 < 32 ? 32 : r->rows + 16); + struct reflow_out *out = grow(r->out_buf, (size_t)cap * sizeof(struct reflow_out)); + if (!out) + return false; + r->out = out; + struct cell *cells = grow(r->cells_buf, (size_t)cap * (size_t)r->cols * sizeof(struct cell)); + if (!cells) + return false; + r->out_cells = cells; + r->out_cap = cap; + } + int idx = r->count++; + struct reflow_out *dst = &r->out[idx]; + struct cell *dst_cells = reflow_out_row_cells(r, idx); + int len = r->x > r->cols ? r->cols : r->x; + memcpy(dst_cells, r->cells, (size_t)len * sizeof(struct cell)); + if (len < r->cols) + fill_cells(dst_cells + len, r->cols - len, TERM_COLOR_FG, TERM_COLOR_BG); + dst->len = len; + dst->wrapped = wrapped; + r->x = 0; + return true; +} + +static bool reflow_append_span(struct reflow_target *r, const struct cell *cells, + int len, int cursor, int saved) +{ + for (int i = 0; i < len;) { + if (r->x == r->cols) { + if (!reflow_target_end_row(r, true)) + return false; + continue; + } + int room = r->cols - r->x; + int take = len - i < room ? len - i : room; + if (i + take < len && (cells[i + take].flags & CELL_TRAIL)) { + if (--take == 0) { + r->cells[r->x] = (struct cell){' ', TERM_COLOR_FG, TERM_COLOR_BG, CELL_WRAP_PAD, 0}; + r->x++; + if (!reflow_target_end_row(r, true)) + return false; + continue; + } + } + if (cursor >= i && cursor < i + take) { + r->cursor_x = r->x + (cursor - i); + r->cursor_y = r->count; + r->cursor_set = true; + r->pending_wrap = false; + } + if (saved >= i && saved < i + take) { + r->saved_x = r->x + (saved - i); + r->saved_y = r->count; + r->saved_set = true; + } + memcpy(r->cells + r->x, cells + i, (size_t)take * sizeof(*cells)); + r->x += take; + i += take; + } + return true; +} + +static bool reflow_append_wide(struct reflow_target *r, const struct cell *cells, int len, + int cursor, int saved) +{ + for (int i = 0; i < len;) { + if (cells[i].flags & (CELL_WRAP_PAD | CELL_TRAIL)) { + int at = (cells[i].flags & CELL_TRAIL) && r->x > 0 ? r->x - 1 : r->x; + if (cursor == i) { + r->cursor_x = at; + r->cursor_y = r->count; + r->cursor_set = true; + r->pending_wrap = false; + } + if (saved == i) { + r->saved_x = at; + r->saved_y = r->count; + r->saved_set = true; + } + i++; + continue; + } + int source = (cells[i].flags & CELL_WIDE) && i + 1 < len && + (cells[i + 1].flags & CELL_TRAIL) ? 2 : 1; + int n = (cells[i].flags & CELL_WIDE) && r->cols > 1 ? 2 : 1; + if (r->x + n > r->cols) { + if (r->x < r->cols) { + r->cells[r->x] = (struct cell){' ', TERM_COLOR_FG, TERM_COLOR_BG, CELL_WRAP_PAD, 0}; + r->x++; + } + if (!reflow_target_end_row(r, true)) + return false; + } + if (cursor >= i && cursor < i + source) { + r->cursor_x = r->x + (cursor - i < n ? cursor - i : 0); + r->cursor_y = r->count; + r->cursor_set = true; + r->pending_wrap = false; + } + if (saved >= i && saved < i + source) { + r->saved_x = r->x + (saved - i < n ? saved - i : 0); + r->saved_y = r->count; + r->saved_set = true; + } + write_cell_span(r->cells, r->x, cells[i], n); + r->x += n; + i += source; + } + return true; +} + +static bool reflow_append(struct reflow_target *r, const struct cell *cells, int len, + int cursor, int saved) +{ + bool has_wide = false; + for (int i = 0; i < len; i++) { + if (cells[i].flags & (CELL_WIDE | CELL_TRAIL | CELL_WRAP_PAD)) { + has_wide = true; + break; + } + } + bool ok = has_wide ? reflow_append_wide(r, cells, len, cursor, saved) + : reflow_append_span(r, cells, len, cursor, saved); + if (!ok) + return false; + + if (cursor == len) { + r->cursor_x = r->x == r->cols ? r->cols - 1 : r->x; + r->cursor_y = r->count; + r->pending_wrap = (r->x == r->cols); + r->cursor_set = true; + } + if (saved == len) { + r->saved_x = r->x == r->cols ? r->cols - 1 : r->x; + r->saved_y = r->count; + r->saved_set = true; + } + return true; +} + +/* Move up to max_pull rows from the end of history to the top of dst. A line + * that does not fit whole leaves its first rows in history, still open. */ +static int pull_history_rows(struct term *t, struct screen *dst_scr, int cols, int max_pull) +{ + struct hist *h = &t->hist; + int pull = 0; + for (uint64_t id = hist_end_id(h); pull < max_pull && id > hist_first_id(h); id--) + pull += term_hist_line_rows(h, id - 1, cols); + if (pull > max_pull) + pull = max_pull; + + int slot = pull; + while (slot > 0) { + uint64_t id = hist_end_id(h) - 1; + int line_rows = term_hist_line_rows(h, id, cols); + bool open = hist_is_open(h); + int take = slot < line_rows ? slot : line_rows; + int first = line_rows - take; + struct cell *pull_buf = grow(&t->resize.pull, (size_t)take * (size_t)cols * sizeof(*pull_buf)); + int *pull_lens = grow(&t->resize.lens, (size_t)take * sizeof(*pull_lens)); + if (pull_buf && pull_lens) + term_hist_decode_segments(h, id, first, take, cols, pull_buf, pull_lens, t->gstore); + for (int seg = line_rows - 1; seg >= first; seg--) { + struct srow *dst = &dst_scr->slot[--slot]; + int idx = seg - first; + int got = (pull_buf && pull_lens) ? pull_lens[idx] : 0; + if (pull_buf) + memcpy(dst->cells, pull_buf + (size_t)idx * cols, (size_t)cols * sizeof(struct cell)); + bool wrapped = seg + 1 < line_rows || open; + if (got < cols) { + fill_cells(dst->cells + got, cols - got, TERM_COLOR_FG, TERM_COLOR_BG); + if (wrapped) { + for (int x = got; x < cols; x++) + dst->cells[x].flags |= CELL_WRAP_PAD; + got = cols; + } + } + dst->kind = ROW_CELLS; + dst->len = (uint16_t)got; + dst->flags = (wrapped ? RF_WRAPPED : 0) | RF_TAIL_PADDED; + dst->version = 1; + dst->uid = ++t->next_row_uid; + } + if (first == 0) { + term_hist_pop_line(h, NULL, 0, t->gstore); + continue; + } + struct cell *rest = grow(&t->resize.rest, (size_t)first * (size_t)cols * sizeof(*rest)); + int *lens = grow(&t->resize.lens, (size_t)first * sizeof(*lens)); + if (rest && lens) + term_hist_decode_segments(h, id, 0, first, cols, rest, lens, t->gstore); + term_hist_pop_line(h, NULL, 0, t->gstore); + if (rest && lens) { + for (int seg = 0; seg < first; seg++) { + struct cell *row = rest + (size_t)seg * cols; + if (lens[seg] < cols) + fill_cells(row + lens[seg], cols - lens[seg], TERM_COLOR_FG, TERM_COLOR_BG); + term_hist_push_row(h, row, cols, true, t->gstore); + } + } + } + return pull; +} + +/* The spare screen, set up as cols x rows, for a resize to build into. */ +static bool resize_target(struct term *t, struct screen *out, int cols, int rows, + const struct screen *from) +{ + *out = t->spare; + t->spare = (struct screen){ 0 }; + if (!screen_setup(out, cols, rows, &t->next_row_uid)) { + t->spare = *out; + return false; + } + out->top_id = from->top_id; + out->top_seg = from->top_seg; + return true; +} + +static void copy_srow_resized(struct srow *dst, struct srow *src, int old_cols, int cols) +{ + int new_len = src->len < cols ? src->len : cols; + if (src->kind == ROW_TEXT && src->len <= cols) { + /* The text fits the new width: copy it into the new buffers. */ + struct text_row *d = &dst->text; + const struct text_row *s = &src->text; + memcpy(d->text, s->text, s->text_len); + memcpy(d->runs.data, s->runs.data, s->runs.len); + d->text_len = s->text_len; + d->cells = s->cells; + d->wide = s->wide; + d->fill_to = s->fill_to < cols ? s->fill_to : (uint16_t)cols; + d->fill = s->fill; + d->flags = s->flags; + d->start = s->start; + d->runs.len = s->runs.len; + d->runs.base = s->runs.base; + d->runs.pen = s->runs.pen; + d->runs.count = s->runs.count; + dst->kind = ROW_TEXT; + dst->flags = (src->flags & (RF_WRAPPED | RF_WIDE)) | RF_TAIL_PADDED; + } else { + srow_settle(src, old_cols); + if (new_len > 0) + memcpy(dst->cells, src->cells, (size_t)new_len * sizeof(struct cell)); + dst->kind = ROW_CELLS; + dst->flags = (src->flags & (RF_WRAPPED | RF_WIDE | RF_CLUSTER)) | RF_TAIL_PADDED; + } + dst->len = (uint16_t)new_len; + dst->uid = src->uid; + dst->version = src->version + 1; +} + +static bool term_resize_internal(struct term *t, int cols, int rows) +{ + term_gstore_pin(t->gstore); + + if (t->alternate || cols == t->scr.cols) { + struct screen *active_scr = term_active_screen(t); + int old_cols = active_scr->cols; + int old_rows = active_scr->rows; + + if (rows < old_rows && t->cursor_y >= rows) { + int shift = t->cursor_y - rows + 1; + for (int i = 0; i < shift; i++) + term_screen_scroll_up(t); + t->cursor_y -= shift; + t->saved.y = t->saved.y >= shift ? t->saved.y - shift : 0; + } + + struct screen new_scr; + if (!resize_target(t, &new_scr, cols, rows, active_scr)) { + term_gstore_unpin(t->gstore); + return false; + } + + int pull = 0; + if (rows > old_rows && !t->alternate) + pull = pull_history_rows(t, &new_scr, cols, rows - old_rows); + + int y = pull; + for (int i = 0; i < old_rows && y < rows; i++, y++) { + struct srow *old_r = screen_get_row(active_scr, i); + copy_srow_resized(&new_scr.slot[y], old_r, old_cols, cols); + } + int first_new = y; + + t->spare = *active_scr; + *active_scr = new_scr; + t->cols = cols; + t->rows = rows; + + if (rows != old_rows) { + for (y = first_new; y < rows; y++) + erase_row(t, phys_row(t, y)); + t->cursor_y += pull; + t->saved.y += pull; + if (!t->alternate && t->prompt_y >= 0) + t->prompt_y += pull; + } + + if (cols != old_cols) + t->pending_wrap = false; + if (t->cursor_x >= cols) t->cursor_x = cols - 1; + if (t->cursor_y >= rows) t->cursor_y = rows - 1; + if (t->saved.x >= cols) t->saved.x = cols - 1; + if (t->saved.y >= rows) t->saved.y = rows - 1; + if (!t->alternate && t->prompt_y >= rows) { + t->prompt_y = -1; + t->has_prompt = false; + } + + t->scroll_top = 0; + t->scroll_bottom = rows - 1; + t->left_margin = 0; + t->right_margin = cols - 1; + term_hist_view_rewrap(&t->view, &t->hist, cols); + t->full_gen++; + term_gstore_unpin(t->gstore); + return true; + } + + int old_cols = t->scr.cols; + int old_rows = t->scr.rows; + + struct screen new_scr; + if (!resize_target(t, &new_scr, cols, rows, &t->scr)) { + term_gstore_unpin(t->gstore); + return false; + } + + int active = old_rows; + while (active > t->cursor_y + 1 && active > t->saved.y + 1) { + struct srow *sr = screen_get_row(&t->scr, active - 1); + if (sr->len || (sr->flags & RF_WRAPPED)) + break; + active--; + } + + struct reflow_target r = { + .cols = cols, + .rows = rows, + .cells = grow(&t->resize.row, (size_t)cols * sizeof(struct cell)), + .out_buf = &t->resize.rows, + .cells_buf = &t->resize.cells, + }; + if (!r.cells) { + t->spare = new_scr; + term_gstore_unpin(t->gstore); + return false; + } + /* The open history line reflows with the screen. It is read here and + * removed only after the reflow succeeds. The prompt line never rewraps. */ + if (hist_is_open(&t->hist) && t->prompt_y != 0) { + uint64_t n = term_hist_line_cells(&t->hist, hist_end_id(&t->hist) - 1); + struct cell *pulled = n ? grow(&t->resize.pull, n * sizeof(*pulled)) : NULL; + if (n && !pulled) + goto reflow_failed; + int got = term_hist_peek_last_line(&t->hist, pulled, pulled ? (int)n : 0, t->gstore); + if (pulled && got > 0 && !reflow_append(&r, pulled, got, -1, -1)) + goto reflow_failed; + } + + size_t line_cap = (size_t)old_cols + (size_t)t->saved.x + 2; + struct cell *line_buf = grow(&t->resize.line, line_cap * sizeof(struct cell)); + + for (int y = 0; y < active; y++) { + struct srow *src = screen_get_row(&t->scr, y); + int len = src->len; + bool wrapped = (src->flags & RF_WRAPPED) != 0; + + if (src->kind != ROW_CELLS) + srow_settle(src, old_cols); + const struct cell *cells = src->cells; + + if (t->prompt_y >= 0 && y >= t->prompt_y) { + if (r.x && !reflow_target_end_row(&r, false)) + goto reflow_failed; + if (y == t->prompt_y) { + r.prompt_y = r.count; + r.prompt_set = true; + } + if (y == t->cursor_y) { + r.cursor_y = r.count; + r.cursor_x = t->cursor_x < cols ? t->cursor_x : cols - 1; + r.pending_wrap = false; + r.cursor_set = true; + } + if (y == t->saved.y) { + r.saved_y = r.count; + r.saved_x = t->saved.x < cols ? t->saved.x : cols - 1; + r.saved_set = true; + } + int keep = len > cols ? cols : len; + memcpy(r.cells, cells, (size_t)keep * sizeof(struct cell)); + r.x = keep; + if (!reflow_target_end_row(&r, wrapped)) + goto reflow_failed; + continue; + } + + if (wrapped && len < old_cols) + len = old_cols; + if (y == t->cursor_y) { + int pos = t->cursor_x + (t->pending_wrap ? 1 : 0); + if (len < pos) + len = pos; + } + if (y == t->saved.y && len <= t->saved.x) + len = t->saved.x + 1; + + if (len > src->len && src->len < old_cols && !(src->flags & RF_TAIL_PADDED)) { + fill_cells(src->cells + src->len, old_cols - src->len, + TERM_COLOR_FG, TERM_COLOR_BG); + src->flags |= RF_TAIL_PADDED; + } + + const struct cell *in_cells = cells; + if (len > old_cols && line_buf && (size_t)len <= line_cap) { + memcpy(line_buf, cells, (size_t)old_cols * sizeof(struct cell)); + fill_cells(line_buf + old_cols, len - old_cols, TERM_COLOR_FG, TERM_COLOR_BG); + in_cells = line_buf; + } else if (len > old_cols) { + len = old_cols; + } + + int cur_x = (y == t->cursor_y) ? t->cursor_x + (t->pending_wrap ? 1 : 0) : -1; + int sav_x = (y == t->saved.y) ? t->saved.x : -1; + if (!reflow_append(&r, in_cells, len, cur_x, sav_x)) + goto reflow_failed; + + if ((!wrapped || y + 1 == active) && !reflow_target_end_row(&r, false)) + goto reflow_failed; + } + if (r.x && !reflow_target_end_row(&r, false)) + goto reflow_failed; + + /* The reflow succeeded. From here the history and the screen change, and + * nothing below can fail. */ + if (hist_is_open(&t->hist)) { + if (t->prompt_y == 0) + term_hist_close_open_line(&t->hist); + else + term_hist_pop_line(&t->hist, NULL, 0, t->gstore); + } + term_hist_set_width(&t->hist, cols); + + int visible = rows - (cols > old_cols ? old_rows - active : 0); + if (visible < 1) + visible = 1; + + int screen_start = 0; + int pull = 0; + if (r.count > visible) { + screen_start = r.count - visible; + if (screen_start > r.cursor_y) + screen_start = r.cursor_y; + for (int i = 0; i < screen_start; i++) { + int push_len = (r.out[i].wrapped && r.out[i].len < cols) ? cols : r.out[i].len; + term_hist_push_row(&t->hist, reflow_out_row_cells(&r, i), push_len, + r.out[i].wrapped, t->gstore); + } + } else if (r.count < visible && t->hist.lines > 0) { + int want = visible - r.count; + if (want > rows - r.count) + want = rows - r.count; + pull = pull_history_rows(t, &new_scr, cols, want); + } + + int out_idx = 0; + for (int i = screen_start; i < r.count && (pull + out_idx) < rows; i++) { + struct srow *dst = &new_scr.slot[pull + out_idx++]; + int len = r.out[i].len; + memcpy(dst->cells, reflow_out_row_cells(&r, i), (size_t)len * sizeof(struct cell)); + if (len < cols) + fill_cells(dst->cells + len, cols - len, TERM_COLOR_FG, TERM_COLOR_BG); + dst->kind = ROW_CELLS; + dst->len = (uint16_t)len; + dst->flags = (r.out[i].wrapped ? RF_WRAPPED : 0) | RF_TAIL_PADDED; + dst->version = 1; + dst->uid = ++t->next_row_uid; + } + + t->spare = t->scr; + t->scr = new_scr; + t->cols = cols; + t->rows = rows; + t->cursor_x = r.cursor_x < cols ? r.cursor_x : cols - 1; + t->cursor_y = (r.cursor_y - screen_start) + pull; + if (t->cursor_y < 0) t->cursor_y = 0; + if (t->cursor_y >= rows) t->cursor_y = rows - 1; + t->pending_wrap = r.pending_wrap && t->autowrap; + + if (t->prompt_y >= 0) { + int new_prompt = (r.prompt_y - screen_start) + pull; + if (!r.prompt_set || new_prompt >= rows) { + t->prompt_y = -1; + t->has_prompt = false; + } else { + t->prompt_y = new_prompt > 0 ? new_prompt : 0; + t->has_prompt = true; + } + } + + if (r.saved_set && (r.saved_y - screen_start) + pull < rows) { + t->saved.x = r.saved_x < cols ? r.saved_x : cols - 1; + int new_saved_y = (r.saved_y - screen_start) + pull; + t->saved.y = new_saved_y < 0 ? 0 : new_saved_y; + } else { + t->saved.y = rows - 1; + } + if (t->saved.x >= cols) t->saved.x = cols - 1; + if (t->saved.y >= rows) t->saved.y = rows - 1; + + t->scroll_top = 0; + t->scroll_bottom = rows - 1; + t->left_margin = 0; + t->right_margin = cols - 1; + term_hist_view_rewrap(&t->view, &t->hist, cols); + t->full_gen++; + term_gstore_unpin(t->gstore); + return true; + +reflow_failed: + t->spare = new_scr; + term_gstore_unpin(t->gstore); + return false; +} + +bool term_resize(struct term *t, struct term_vector2i size) +{ + if (!t) + return false; + int cols = size.x; + int rows = size.y; + if (cols < 1 || rows < 1) + return false; + if (cols > TERM_MAX_COLS) + cols = TERM_MAX_COLS; + term_write_begin(t); + if (cols == t->cols && rows == t->rows) { + term_write_end(t); + return true; + } + + if (cols > t->tabs_cap) { + int new_cap = cols + 64; + uint8_t *new_tabs = realloc(t->tabs, (size_t)new_cap); + if (!new_tabs) { + term_write_end(t); + return false; + } + memset(new_tabs + t->tabs_cap, 0, (size_t)(new_cap - t->tabs_cap)); + int first_tab = (t->tabs_cap + 7) & ~7; + if (first_tab == 0) + first_tab = 8; + for (int c = first_tab; c < new_cap; c += 8) + new_tabs[c] = 1; + t->tabs = new_tabs; + t->tabs_cap = new_cap; + } + if (cols > t->view_scratch_cap) { + int new_cap = cols + 64; + struct cell *new_scratch = realloc(t->view_row_scratch, (size_t)new_cap * sizeof(struct cell)); + if (!new_scratch) { + term_write_end(t); + return false; + } + memset(new_scratch + t->view_scratch_cap, 0, (size_t)(new_cap - t->view_scratch_cap) * sizeof(struct cell)); + t->view_row_scratch = new_scratch; + t->view_scratch_cap = new_cap; + } + + bool ok = term_resize_internal(t, cols, rows); + term_write_end(t); + return ok; +} diff --git a/src/screen.h b/src/screen.h new file mode 100644 index 0000000..0110049 --- /dev/null +++ b/src/screen.h @@ -0,0 +1,453 @@ +#ifndef TERM_SCREEN_H +#define TERM_SCREEN_H + +#include <assert.h> +#include "compat.h" +#include "internal.h" +#include "kernels.h" +#include "simd.h" + +void term_screen_scroll_region(struct term *t, int top, int bottom, int count, bool down, bool history); +void term_screen_set_alternate(struct term *t, bool enabled); +void term_screen_erase(struct term *t, int mode); +void term_screen_erase_all(struct term *t); +void term_screen_erase_line(struct term *t, int mode); +void term_screen_delete_chars(struct term *t, int count); +void term_screen_insert_chars(struct term *t, int count); + +const struct cell *term_ring_row(const struct term *t, int row); +const struct cell *term_screen_view_row(const struct term *t, int y); +uint64_t term_screen_view_row_key(const struct term *t, int y); +/* The selected columns [c0, c1] of each view row; c1 < c0 when none. */ +void term_selection_view_spans(const struct term *t, int *c0, int *c1); + +int term_screen_history_rows(const struct term *t); + +/* Rows of history above the live screen. The view owns the offset. */ +int term_view_offset(const struct term *t); + +static inline struct screen *term_active_screen(const struct term *t) +{ + return (struct screen *)&t->scr; +} + +static inline bool screen_has_history(const struct term *t) +{ + return !t->alternate && t->hist.line_limit > 0; +} + +static inline int phys_row(const struct term *t, int y) +{ + int p = t->scr.top + y; + return p >= t->scr.rows ? p - t->scr.rows : p; +} + +static inline void init_ascii_text_row(struct text_row *tr, uint8_t *text, uint16_t len) +{ + tr->text = text; + tr->text_len = tr->cells = len; + tr->wide = tr->fill_to = 0; + tr->flags = HLINE_NARROW; + tr->runs.len = 0; + tr->runs.count = len; +#if defined(__x86_64__) || defined(_M_X64) || defined(__aarch64__) + static_assert(sizeof(struct pen_attr) == 12, + "a 16-byte store writes one pen and 4 spare bytes"); + static_assert(offsetof(struct hist_runs, base) + 12 == offsetof(struct hist_runs, pen), + "the spare bytes of the store at base land on pen"); + static_assert(offsetof(struct hist_runs, pen) + 16 <= sizeof(struct hist_runs), + "the store at pen stays inside hist_runs"); + static_assert(offsetof(struct text_row, start) + 16 <= offsetof(struct text_row, runs), + "the store at start stays before runs"); +#endif + /* The extra lanes cover unused fill data and run padding. */ +#if defined(__x86_64__) || defined(_M_X64) + __m128i pen = _mm_setr_epi32(TERM_COLOR_FG, TERM_COLOR_BG, 0, TERM_COLOR_FG); + _mm_storeu_si128((__m128i *)&tr->start, pen); + _mm_storeu_si128((__m128i *)&tr->runs.base, pen); + _mm_storeu_si128((__m128i *)&tr->runs.pen, pen); +#elif defined(__aarch64__) + uint32x4_t pen = { TERM_COLOR_FG, TERM_COLOR_BG, 0, TERM_COLOR_FG }; + vst1q_u32((uint32_t *)&tr->start, pen); + vst1q_u32((uint32_t *)&tr->runs.base, pen); + vst1q_u32((uint32_t *)&tr->runs.pen, pen); +#else + tr->start = tr->runs.base = tr->runs.pen = pen_attr_default(); +#endif +} + +static inline void srow_materialize_text(struct srow *row) +{ + if (row->kind == ROW_ASCII) { + init_ascii_text_row(&row->text, row->text.text, row->len); + row->kind = ROW_TEXT; + } +} + +static inline struct srow *screen_get_row(const struct screen *s, int y) +{ + int p = s->top + y; + struct srow *row = &s->slot[p >= s->rows ? p - s->rows : p]; + srow_materialize_text(row); + return row; +} + +/* The row at y, or NULL when y is outside the screen. A view row that + * resolves to no screen row must not index the ring. */ +static inline struct srow *screen_row_at(const struct screen *s, int y) +{ + if (y < 0 || y >= s->rows) + return NULL; + return screen_get_row(s, y); +} + +static inline struct srow *term_row(const struct term *t, int y) +{ + int p = t->scr.top + y; + struct srow *row = &t->scr.slot[p >= t->scr.rows ? p - t->scr.rows : p]; + srow_materialize_text(row); + return row; +} + +static inline struct srow *term_screen_row(const struct term *t, int y) +{ + return term_row(t, y); +} + +/* Write the cells of a text row to out: its text, then its fill blanks. + * Return the cells written. */ +static inline int text_row_decode(const struct srow *row, struct cell *out, int cols) +{ + srow_materialize_text((struct srow *)row); + const struct text_row *tr = &row->text; + int n = term_hist_text_decode(tr, out, cols, NULL); + int fill = tr->fill_to < cols ? tr->fill_to : cols; + if (n < fill) { + fill_cells(out + n, fill - n, tr->fill.fg, tr->fill.bg); + n = fill; + } + return n; +} + +/* The cells of a text row, cols wide, for a reader. */ +static inline void text_row_cells(const struct srow *row, struct cell *out, int cols) +{ + int n = text_row_decode(row, out, cols); + if (n < cols) + fill_cells(out + n, cols - n, TERM_COLOR_FG, TERM_COLOR_BG); +} + +/* Turn a text row into a cells row. Only the write core does this. Its + * cells past the text are still blank when RF_TAIL_PADDED says so. */ +static inline void srow_settle(struct srow *row, int cols) +{ + srow_materialize_text(row); + if (row->kind == ROW_TEXT) { + int n = text_row_decode(row, row->cells, cols); + if (n < cols && !(row->flags & RF_TAIL_PADDED)) + fill_cells(row->cells + n, cols - n, TERM_COLOR_FG, TERM_COLOR_BG); + row->flags |= RF_TAIL_PADDED; + row->kind = ROW_CELLS; + } +} + +/* + * The cursor row as a row that text can append to: a text row with the + * cursor at its end, or an empty row on the bottom line of a full-screen + * region with history, which becomes a text row. Rows there scroll into + * history, so they keep the history format. NULL when neither holds. The + * caller has handled a pending wrap, insert mode, and margins. + */ +static inline struct srow *text_append_row(struct term *t) +{ + struct srow *row = term_row(t, t->cursor_y); + if (row->kind == ROW_TEXT) { + text_row_unshare(&row->text); + if (t->cursor_x != row->text.cells) + return NULL; + if (row->text.cells) + return row; + } else if (row->len != 0 || t->cursor_x != 0 || t->cursor_y != t->rows - 1 || + t->scroll_top != 0 || t->scroll_bottom != t->rows - 1 || !screen_has_history(t)) { + return NULL; + } + /* A row that continues a text row starts from that row's last pen, so + * history can copy its runs. */ + struct pen_attr start = pen_attr_default(); + if (t->cursor_y > 0) { + const struct srow *prev = term_row(t, t->cursor_y - 1); + if (prev->kind == ROW_TEXT && (prev->flags & RF_WRAPPED)) + start = text_row_last_pen(&prev->text); + } + if (row->kind == ROW_TEXT) { + uint32_t sl = pen_sl(&start); + pen_set(&row->text.start, start.fg, start.bg, sl); + pen_set(&row->text.runs.base, start.fg, start.bg, sl); + pen_set(&row->text.runs.pen, start.fg, start.bg, sl); + return row; + } + row->kind = ROW_TEXT; + text_row_start(&row->text, &start); + return row; +} + +/* Take nbytes bytes that a writer put at the end of the row text: chars + * characters in cells columns, with the pen fg, bg, sl (flags | link << 16). + * Runs count characters. */ +static inline void text_commit(struct srow *row, int nbytes, int chars, int cells, uint8_t hflags, + uint32_t fg, uint32_t bg, uint32_t sl) +{ + struct text_row *tr = &row->text; + tr->text_len = (uint16_t)(tr->text_len + nbytes); + tr->cells = (uint16_t)(tr->cells + cells); + tr->wide = (uint16_t)(tr->wide + cells - chars); + tr->flags = (uint8_t)((tr->flags & hflags & HLINE_NARROW) | + ((tr->flags | hflags) & ~HLINE_NARROW) | + ((sl >> 16) ? HLINE_HAS_LINK : 0)); + hist_runs_add_pen(&tr->runs, fg, bg, sl, (uint32_t)chars); + if (tr->cells > row->len) + row->len = tr->cells; + row->version++; + if (hflags & HLINE_HAS_WIDE) + row->flags |= RF_WIDE; +} + +static inline void text_append(struct srow *row, const uint8_t *bytes, int nbytes, int chars, + int cells, uint8_t hflags, uint32_t fg, uint32_t bg, uint32_t sl) +{ + bytes_copy_small(row->text.text + row->text.text_len, (const char *)bytes, nbytes); + text_commit(row, nbytes, chars, cells, hflags, fg, bg, sl); +} + +static inline struct cell *row_cells(const struct term *t, int p) +{ + struct srow *row = &t->scr.slot[p]; + if (row->kind != ROW_CELLS) + srow_settle(row, t->cols); + return row->cells; +} + +/* Screen rows carry no line ids. A row's logical line follows from the + * history tail and the RF_WRAPPED chain above it, so a scroll, a wrap change, + * or a resize can never leave a stale id behind. */ + +/* The logical line of screen row 0, and the row's segment in it. */ +static inline struct line_pos screen_top_line(const struct term *t) +{ + if (!screen_has_history(t)) + return (struct line_pos){ t->scr.top_id, t->scr.top_seg }; + uint64_t end = hist_end_id(&t->hist); + if (!hist_is_open(&t->hist)) + return (struct line_pos){ end, 0 }; + return (struct line_pos){ end - 1, (uint32_t)term_hist_line_rows(&t->hist, end - 1, t->cols) }; +} + +static inline struct line_pos line_pos_next(struct line_pos p, const struct srow *row) +{ + return (row->flags & RF_WRAPPED) ? (struct line_pos){ p.id, p.seg + 1 } + : (struct line_pos){ p.id + 1, 0 }; +} + +/* The oldest line that history or the screen still holds. */ +static inline uint64_t first_retained_line(const struct term *t) +{ + if (screen_has_history(t) && t->hist.lines) + return hist_first_id(&t->hist); + return screen_top_line(t).id; +} + +static inline void row_set_wrapped(struct srow *row, bool wrapped) +{ + row->flags = wrapped ? (row->flags | RF_WRAPPED) : (row->flags & ~RF_WRAPPED); +} + +static inline void clear_wide_edges(struct cell *row, int from, int to, int len) +{ + if (from >= to) return; + if (from > 0 && from < len && (row[from].flags & CELL_TRAIL)) + fill_cells(row + from - 1, 1, row[from - 1].fg, row[from - 1].bg); + if (to < len && (row[to].flags & CELL_TRAIL)) + fill_cells(row + to, 1, row[to].fg, row[to].bg); +} + +static inline void repair_wide_row(struct cell *row, int from, int to, int cols, int len) +{ + if (from < 0) from = 0; + if (to > len) to = len; + if (to > cols) to = cols; + for (int x = from; x < to; x++) { + if ((row[x].flags & CELL_TRAIL) && + (x == 0 || !(row[x - 1].flags & CELL_WIDE))) + fill_cells(row + x, 1, row[x].fg, row[x].bg); + if ((row[x].flags & CELL_WIDE) && cols > 1 && + (x + 1 == cols || x + 1 >= len || !(row[x + 1].flags & CELL_TRAIL))) + fill_cells(row + x, 1, row[x].fg, row[x].bg); + } +} + +static inline bool using_default_colors(const struct term *t) +{ + return t->cur_fg == TERM_COLOR_FG && t->cur_bg == TERM_COLOR_BG; +} + +static inline void mark_row_blank(struct term *t, int p, uint32_t fg, uint32_t bg) +{ + struct srow *row = &t->scr.slot[p]; + uint8_t tail_padded = ((row->kind == ROW_CELLS && row->len == 0) || row->kind == ROW_TEXT) + ? (row->flags & RF_TAIL_PADDED) : 0; + row->kind = ROW_CELLS; + row->len = 0; + row->flags = tail_padded; + row->version++; + (void)fg; + (void)bg; +} + +static inline void row_fill_gap(struct term *t, int p, struct cell *row, int x) +{ + struct screen *s = term_active_screen(t); + struct srow *r = &s->slot[p]; + if (x > r->len) { + if (!(r->flags & RF_TAIL_PADDED)) + fill_cells(row + r->len, x - r->len, TERM_COLOR_FG, TERM_COLOR_BG); + r->len = (uint16_t)x; + } +} + +static inline void term_row_fill_gap(struct srow *row, int x) +{ + if (x > row->len) { + if (!(row->flags & RF_TAIL_PADDED)) + fill_cells(row->cells + row->len, x - row->len, TERM_COLOR_FG, TERM_COLOR_BG); + row->len = (uint16_t)x; + } +} + +static inline void write_cell_span(struct cell *row, int x, + struct cell cell, int span) +{ + row[x] = cell; + if (span == 2) { + row[x + 1] = cell; + row[x + 1].c = 0; + row[x + 1].flags = (uint16_t)((cell.flags & ~CELL_WIDE) | CELL_TRAIL); + } +} + +static inline void erase_span(struct term *t, int p, int from, int to) +{ + bool pristine = using_default_colors(t); + struct screen *s = term_active_screen(t); + struct srow *row = &s->slot[p]; + int len = row->len; + if (to > TERM_MAX_COLS) to = TERM_MAX_COLS; + if (pristine && to > len) to = len; + if (from >= to) return; + + if (pristine && from == 0 && to >= len) { + mark_row_blank(t, p, t->cur_fg, t->cur_bg); + return; + } + struct cell *cells = row_cells(t, p); + if (from > 0 && from < len && (cells[from].flags & CELL_TRAIL)) from--; + if (to < len && (cells[to].flags & CELL_TRAIL)) to++; + if (to >= t->cols) + row_set_wrapped(row, false); + + if (!pristine) { + row_fill_gap(t, p, cells, from); + fill_cells(cells + from, to - from, t->cur_fg, t->cur_bg); + if (to > len) { + row->len = (uint16_t)to; + if (to < t->cols) + row->flags &= ~RF_TAIL_PADDED; + } + row->version++; + return; + } + if (to < len) { + fill_cells(cells + from, to - from, t->cur_fg, t->cur_bg); + row->version++; + return; + } + if (from == 0) { + mark_row_blank(t, p, t->cur_fg, t->cur_bg); + } else { + row->len = (uint16_t)from; + row->flags &= ~RF_TAIL_PADDED; + row->version++; + } +} + +static inline int erase_width(const struct term *t, int p) +{ + const struct screen *s = term_active_screen(t); + return s->slot[p].len > t->cols ? s->slot[p].len : t->cols; +} + +static inline void erase_row(struct term *t, int p) +{ + erase_span(t, p, 0, erase_width(t, p)); +} + +static inline void clamp_selection_after_scroll(struct term *t) +{ + if (TERM_EXPECT(!t->selection, 1)) + return; + uint64_t first_id = first_retained_line(t); + if (t->sel_end.line_id < first_id) { + t->selection = false; + } else if (t->sel_start.line_id < first_id) { + t->sel_start.line_id = first_id; + t->sel_start.cell_off = 0; + } +} + +/* Move a row that leaves the top of the normal screen into history. */ +void term_screen_push_row(struct term *t, struct srow *row); + +/* Count a row that leaves the top of a screen without history. */ +static inline void screen_drop_top(struct screen *s, const struct srow *row) +{ + struct line_pos p = line_pos_next((struct line_pos){ s->top_id, s->top_seg }, row); + s->top_id = p.id; + s->top_seg = p.seg; +} + +/* Clear a row that enters at the bottom. With a colored pen it is a text row + * that is all fill: blanks in the pen colors, which text can append over. */ +static inline void screen_new_row(struct term *t, struct srow *row) +{ + row->uid = ++t->next_row_uid; + row->version++; + row->flags = ((row->kind == ROW_CELLS && row->len == 0) || row->kind == ROW_TEXT) + ? (row->flags & RF_TAIL_PADDED) : 0; + if (TERM_EXPECT(using_default_colors(t), 1)) { + row->kind = ROW_CELLS; + row->len = 0; + } else { + row->kind = ROW_TEXT; + text_row_start(&row->text, &(struct pen_attr){ .fg = TERM_COLOR_FG, .bg = TERM_COLOR_BG }); + row->text.fill_to = (uint16_t)t->scr.cols; + row->text.fill = (struct pen_attr){ .fg = t->cur_fg, .bg = t->cur_bg }; + row->len = (uint16_t)t->scr.cols; + } +} + +static inline void term_screen_scroll_up(struct term *t) +{ + struct screen *s = &t->scr; + struct srow *row = &s->slot[s->top]; + if (TERM_EXPECT(screen_has_history(t), 1)) + term_screen_push_row(t, row); + else + screen_drop_top(s, row); + if (TERM_EXPECT(t->prompt_y > 0, 0) && !t->alternate) + t->prompt_y--; + s->top = s->top + 1 == s->rows ? 0 : s->top + 1; + clamp_selection_after_scroll(t); + screen_new_row(t, row); + t->full_gen++; +} + +#endif diff --git a/src/selection.c b/src/selection.c new file mode 100644 index 0000000..6f7336f --- /dev/null +++ b/src/selection.c @@ -0,0 +1,824 @@ +#include "screen.h" +#include "osc/osc.h" +#include "unicode/grapheme.h" +#include "unicode/utf8.h" +#include <stdlib.h> +#include <string.h> + +/* Copies a heap string into buf and frees it. Returns the length without the NUL. */ +static size_t text_out(char *text, char *buf, size_t cap) +{ + size_t n = term_copy_out(text, buf, cap); + free(text); + return n; +} + +static char *link_at_text(const struct term *t, struct term_vector2i cell) +{ + int col = cell.x, row = cell.y; + if (!t || col < 0 || col >= t->cols || row < 0 || row >= t->rows) return NULL; + const char *link = term_osc8_url_at(t, col, row); + if (link) return term_strdup(link); + const struct cell *cells = term_screen_view_row(t, row); + if (!cells) return NULL; + int start = col, end = col; + while (start > 0 && cells[start - 1].c > ' ' && cells[start - 1].c < 127) start--; + while (end < t->cols && cells[end].c > ' ' && cells[end].c < 127) end++; + char *text = malloc((size_t)(end - start) + 1); + if (!text) return NULL; + for (int c = start; c < end; c++) text[c - start] = cells[c].c; + text[end - start] = 0; + size_t scheme = 8; + char *url = strstr(text, "https://"); + if (!url) { + scheme = 7; + url = strstr(text, "http://"); + if (!url) url = strstr(text, "file://"); + } + char *out = NULL; + if (url && col >= start + (int)(url - text)) { + size_t n = strlen(url); + while (n && strchr(".,;:!?)]}>\"'", url[n - 1])) url[--n] = 0; + if (col < start + (int)(url - text) + (int)n && n > scheme) + out = term_strdup(url); + } + free(text); + return out; +} + +size_t term_link_at(const struct term *t, struct term_vector2i cell, char *buf, size_t cap) +{ + if (!t) + return 0; + term_write_begin((struct term *)t); + char *text = link_at_text(t, cell); + term_write_end((struct term *)t); + return text_out(text, buf, cap); +} + +void term_select_clear(struct term *t) +{ + if (t) { + term_write_begin(t); + t->selection = false; + term_write_end(t); + } +} + +static inline int word_class(uint32_t c) +{ + if (c <= ' ') return 0; + if (c < 128) { + static const uint64_t mask[2] = { + 0x03FF000000000000ULL, + 0x07FFFFFE87FFFFFEULL + }; + return (mask[c >> 6] & (1ULL << (c & 63))) ? 1 : 2; + } + return 1; +} + +static inline int sel_point_cmp(struct term_sel_point a, struct term_sel_point b) +{ + if (a.line_id < b.line_id) return -1; + if (a.line_id > b.line_id) return 1; + if (a.cell_off < b.cell_off) return -1; + if (a.cell_off > b.cell_off) return 1; + return 0; +} + +/* + * The lines that one call reads. History holds every row of its closed + * lines and the first rows of the open line. The screen holds the rest; pos + * gives the line and segment of each screen row. + */ +struct lines { + const struct term *t; + int cols; + bool hist; + uint64_t hist_end; + uint64_t first; + struct line_pos *pos; + struct line_pos stack[256]; +}; + +static void lines_init(struct lines *L, const struct term *t) +{ + L->t = t; + L->cols = t->cols; + L->hist = screen_has_history(t); + L->hist_end = L->hist ? hist_end_id(&t->hist) : 0; + L->first = first_retained_line(t); + L->pos = t->rows <= 256 ? L->stack : malloc((size_t)t->rows * sizeof(*L->pos)); + if (!L->pos) + L->pos = L->stack; + int rows = L->pos == L->stack && t->rows > 256 ? 256 : t->rows; + struct line_pos p = screen_top_line(t); + for (int y = 0; y < rows; y++) { + L->pos[y] = p; + p = line_pos_next(p, screen_get_row(term_active_screen(t), y)); + } +} + +static void lines_free(struct lines *L) +{ + if (L->pos != L->stack) + free(L->pos); +} + +static inline int lines_rows(const struct lines *L) +{ + return L->pos == L->stack && L->t->rows > 256 ? 256 : L->t->rows; +} + +static inline bool in_hist(const struct lines *L, uint64_t id) +{ + return L->hist && id >= hist_first_id(&L->t->hist) && id < L->hist_end; +} + +static inline bool is_open_line(const struct lines *L, uint64_t id) +{ + return L->hist && id + 1 == L->hist_end && hist_is_open(&L->t->hist); +} + +static inline int hist_segs(const struct lines *L, uint64_t id) +{ + return in_hist(L, id) ? term_hist_line_rows(&L->t->hist, id, L->cols) : 0; +} + +static int screen_row_of(const struct lines *L, uint64_t id, int seg) +{ + int rows = lines_rows(L); + for (int y = 0; y < rows; y++) + if (L->pos[y].id == id && (int)L->pos[y].seg == seg) + return y; + return -1; +} + +/* Segments of line id across history and the screen. */ +static int line_total_segs(const struct lines *L, uint64_t id) +{ + int segs = hist_segs(L, id); + int rows = lines_rows(L); + for (int y = 0; y < rows; y++) + if (L->pos[y].id == id && (int)L->pos[y].seg + 1 > segs) + segs = (int)L->pos[y].seg + 1; + return segs; +} + +/* The cells of one segment, padded to the width. */ +static const struct cell *line_seg_cells(const struct lines *L, uint64_t id, int seg, + struct cell *scratch, bool *out_wrapped) +{ + const struct term *t = L->t; + struct cell *buf = scratch ? scratch : t->view_row_scratch; + int hrows = hist_segs(L, id); + if (seg < hrows) { + int got = term_hist_decode_segment(&t->hist, id, seg, L->cols, buf, t->gstore); + bool wrapped = seg + 1 < hrows || is_open_line(L, id); + if (got < L->cols) { + fill_cells(buf + got, L->cols - got, TERM_COLOR_FG, TERM_COLOR_BG); + if (wrapped) + for (int x = got; x < L->cols; x++) + buf[x].flags |= CELL_WRAP_PAD; + } + if (out_wrapped) + *out_wrapped = wrapped; + return buf; + } + int y = screen_row_of(L, id, seg); + if (y < 0) + return NULL; + const struct srow *srow = screen_get_row(term_active_screen(t), y); + if (out_wrapped) + *out_wrapped = (srow->flags & RF_WRAPPED) != 0; + if (srow->kind == ROW_CELLS) { + if (srow->len >= L->cols || (srow->flags & RF_TAIL_PADDED)) + return srow->cells; + if (srow->len > 0) + memcpy(buf, srow->cells, (size_t)srow->len * sizeof(struct cell)); + fill_cells(buf + srow->len, L->cols - srow->len, TERM_COLOR_FG, TERM_COLOR_BG); + return buf; + } + text_row_cells(srow, buf, L->cols); + return buf; +} + +/* A line whose segments may not hold exactly cols cells each. */ +static bool line_has_wide(const struct lines *L, uint64_t id) +{ + const struct term *t = L->t; + if (in_hist(L, id)) { + if (term_hist_line_flags(&t->hist, id) & (HLINE_HAS_WIDE | HLINE_HAS_CLUSTER)) + return true; + if (is_open_line(L, id) && + term_hist_line_cells(&t->hist, id) != (uint64_t)hist_segs(L, id) * (uint64_t)L->cols) + return true; + } + int rows = lines_rows(L); + for (int y = 0; y < rows; y++) { + if (L->pos[y].id != id) + continue; + const struct srow *r = screen_get_row(term_active_screen(t), y); + if (r->flags & RF_WIDE) + return true; + if (r->kind == ROW_CELLS && r->len >= L->cols && L->cols > 1 && + (r->cells[L->cols - 1].flags & CELL_WRAP_PAD)) + return true; + } + return false; +} + +static inline int seg_display_cols(int cols, const struct cell *cells, bool wrapped) +{ + if (cells && wrapped && cols > 1 && (cells[cols - 1].flags & CELL_WRAP_PAD)) + return cols - 1; + return cols; +} + +/* The display column where segment seg of line id starts. */ +static uint32_t line_seg_start_off(const struct lines *L, uint64_t id, int seg) +{ + if (seg <= 0) + return 0; + if (!line_has_wide(L, id)) + return (uint32_t)seg * (uint32_t)L->cols; + + const struct term *t = L->t; + int hrows = hist_segs(L, id); + if (in_hist(L, id)) { + if (seg <= hrows) { + uint64_t base = id == t->hist.base_cell_id ? t->hist.base_cell : 0; + return (uint32_t)(term_hist_segment_cell(&t->hist, id, seg, L->cols) - base); + } + } + + uint32_t off = 0; + if (in_hist(L, id) && hrows > 0) { + uint64_t base = id == t->hist.base_cell_id ? t->hist.base_cell : 0; + off = (uint32_t)(term_hist_segment_cell(&t->hist, id, hrows, L->cols) - base); + } + struct cell stack_buf[512]; + struct cell *scratch = L->cols <= 512 ? stack_buf : malloc((size_t)L->cols * sizeof(*scratch)); + for (int s = hrows; s < seg; s++) { + bool wrapped = false; + const struct cell *cells = line_seg_cells(L, id, s, scratch, &wrapped); + off += (uint32_t)seg_display_cols(L->cols, cells, wrapped); + } + if (scratch != stack_buf) + free(scratch); + return off; +} + +/* The line and segment of view row y. */ +static bool view_row_line(const struct lines *L, int y, uint64_t *id, int *seg) +{ + const struct term *t = L->t; + int y_scr; + if (term_hist_view_row_pos(&t->view, &t->hist, L->cols, t->rows, screen_top_line(t), y, + id, seg, &y_scr)) + return true; + if (y_scr >= lines_rows(L)) + y_scr = lines_rows(L) - 1; + *id = L->pos[y_scr].id; + *seg = (int)L->pos[y_scr].seg; + return true; +} + +static struct term_sel_point view_to_sel_point(const struct lines *L, int x, int y) +{ + const struct term *t = L->t; + if (x < 0) x = 0; + if (x >= L->cols) x = L->cols - 1; + if (y < 0) y = 0; + if (y >= t->rows) y = t->rows - 1; + + uint64_t id; + int seg; + if (!view_row_line(L, y, &id, &seg)) + return (struct term_sel_point){ .line_id = L->first }; + const struct cell *cells = line_seg_cells(L, id, seg, NULL, NULL); + if (cells) + while (x > 0 && (cells[x].flags & CELL_WRAP_PAD)) + x--; + return (struct term_sel_point){ + .line_id = id, + .cell_off = line_seg_start_off(L, id, seg) + (uint32_t)x, + }; +} + +static uint32_t line_total_display_cols(const struct lines *L, uint64_t id, int nsegs) +{ + if (nsegs <= 0) + return 0; + uint32_t start = line_seg_start_off(L, id, nsegs - 1); + bool wrapped = false; + const struct cell *cells = line_seg_cells(L, id, nsegs - 1, NULL, &wrapped); + return start + (uint32_t)seg_display_cols(L->cols, cells, wrapped); +} + +static struct term_sel_point sel_point_next(const struct lines *L, struct term_sel_point pt) +{ + int nsegs = line_total_segs(L, pt.line_id); + if (nsegs <= 0) + return pt; + if (pt.cell_off + 1 < (uint64_t)(nsegs - 1) * (L->cols > 1 ? (uint64_t)(L->cols - 1) : 1ULL) || + pt.cell_off + 1 < line_total_display_cols(L, pt.line_id, nsegs)) { + pt.cell_off++; + return pt; + } + uint64_t max_id = L->pos[lines_rows(L) - 1].id; + for (uint64_t next = pt.line_id + 1; next <= max_id; next++) + if (line_total_segs(L, next) > 0) + return (struct term_sel_point){ .line_id = next }; + pt.cell_off++; + return pt; +} + +static struct term_sel_point sel_point_prev(const struct lines *L, struct term_sel_point pt) +{ + if (pt.cell_off > 0) { + pt.cell_off--; + return pt; + } + for (uint64_t prev = pt.line_id; prev-- > L->first; ) { + int nsegs = line_total_segs(L, prev); + if (nsegs <= 0) + continue; + uint32_t cols = line_total_display_cols(L, prev, nsegs); + return (struct term_sel_point){ .line_id = prev, .cell_off = cols > 0 ? cols - 1 : 0 }; + } + return pt; +} + +static void unit_span(const struct lines *L, int x, int y, bool right_half, + enum term_select_unit unit, + struct term_sel_point *first, struct term_sel_point *last) +{ + const struct term *t = L->t; + int cols = L->cols; + if (x < 0) x = 0; + if (x >= cols) x = cols - 1; + if (y < 0) y = 0; + if (y >= t->rows) y = t->rows - 1; + + if (unit == TERM_SELECT_CELL) { + struct term_sel_point pt = view_to_sel_point(L, x, y); + if (right_half) + pt = sel_point_next(L, pt); + *first = *last = pt; + return; + } + if (unit == TERM_SELECT_LINE) { + *first = view_to_sel_point(L, 0, y); + *last = sel_point_next(L, view_to_sel_point(L, cols - 1, y)); + return; + } + + const struct cell *row = term_screen_view_row(t, y); + if (row) + while (x > 0 && (row[x].flags & CELL_WRAP_PAD)) + x--; + int left = x, right = x; + int kind = row ? word_class(row[x].c) : 0; + while (left > 0 && row && word_class(row[left - 1].c) == kind) + left--; + while (right + 1 < cols && row && !(row[right + 1].flags & CELL_WRAP_PAD) && + word_class(row[right + 1].c) == kind) + right++; + *first = view_to_sel_point(L, left, y); + *last = sel_point_next(L, view_to_sel_point(L, right, y)); +} + +/* Clamp the selection to the retained lines. A read does not change it. */ +static bool selection_bounds(const struct lines *L, struct term_sel_point *start, + struct term_sel_point *end) +{ + const struct term *t = L->t; + if (!t->selection || t->sel_end.line_id < L->first) + return false; + *start = t->sel_start; + *end = t->sel_end; + if (start->line_id < L->first) + *start = (struct term_sel_point){ .line_id = L->first }; + return sel_point_cmp(*start, *end) <= 0; +} + +static bool line_locate_cell(const struct lines *L, uint64_t id, uint32_t cell_off, + int *out_seg, int *out_col) +{ + int nsegs = line_total_segs(L, id); + if (nsegs <= 0) + return false; + if (!line_has_wide(L, id)) { + int seg = (int)(cell_off / (uint32_t)L->cols); + if (seg >= nsegs) + return false; + if (out_seg) + *out_seg = seg; + if (out_col) + *out_col = (int)(cell_off % (uint32_t)L->cols); + return true; + } + + int hrows = hist_segs(L, id); + if (in_hist(L, id) && hrows > 0) { + uint32_t last_start = line_seg_start_off(L, id, hrows - 1); + if (cell_off >= last_start) { + if (hrows == nsegs) { + if (cell_off < last_start + (uint32_t)L->cols) { + if (out_seg) + *out_seg = hrows - 1; + if (out_col) + *out_col = (int)(cell_off - last_start); + return true; + } + return false; + } + } else { + uint64_t base = id == L->t->hist.base_cell_id ? L->t->hist.base_cell : 0; + int seg = term_hist_cell_segment(&L->t->hist, id, base + (uint64_t)cell_off, L->cols); + if (seg < hrows) { + uint32_t seg_start = line_seg_start_off(L, id, seg); + if (cell_off >= seg_start) { + if (out_seg) + *out_seg = seg; + if (out_col) + *out_col = (int)(cell_off - seg_start); + return true; + } + } + } + } + + int start_seg = (in_hist(L, id) && hrows > 0 && hrows < nsegs) ? hrows - 1 : hrows; + uint32_t start = line_seg_start_off(L, id, start_seg); + for (int seg = start_seg; seg < nsegs; seg++) { + bool wrapped = false; + const struct cell *cells = line_seg_cells(L, id, seg, NULL, &wrapped); + uint32_t w = (uint32_t)seg_display_cols(L->cols, cells, wrapped); + if (cell_off < start + w) { + if (out_seg) + *out_seg = seg; + if (out_col) + *out_col = (int)(cell_off - start); + return true; + } + start += w; + } + return false; +} + +static void select_range(struct term *t, const struct lines *L, + struct term_sel_point start, struct term_sel_point end) +{ + if (sel_point_cmp(start, end) > 0) { + t->selection = false; + return; + } + int seg = 0, col = 0; + if (line_locate_cell(L, start.line_id, start.cell_off, &seg, &col)) { + const struct cell *cells = line_seg_cells(L, start.line_id, seg, NULL, NULL); + if (cells && col > 0 && (cells[col].flags & CELL_TRAIL)) + start = sel_point_prev(L, start); + } + if (line_locate_cell(L, end.line_id, end.cell_off, &seg, &col)) { + const struct cell *cells = line_seg_cells(L, end.line_id, seg, NULL, NULL); + if (cells && col + 1 < L->cols && (cells[col].flags & CELL_WIDE)) + end = sel_point_next(L, end); + } + t->sel_start = start; + t->sel_end = end; + t->selection = true; +} + +void term_select_start(struct term *t, struct term_vector2i cell, bool right_half, + enum term_select_unit unit) +{ + if (!t) return; + term_write_begin(t); + struct lines L; + lines_init(&L, t); + t->select_unit = unit; + unit_span(&L, cell.x, cell.y, right_half, unit, &t->sel_anchor_first, &t->sel_anchor_last); + t->selection = false; + lines_free(&L); + term_select_extend(t, cell, right_half); + term_write_end(t); +} + +void term_select_extend(struct term *t, struct term_vector2i cell, bool right_half) +{ + if (!t) return; + term_write_begin(t); + struct lines L; + lines_init(&L, t); + struct term_sel_point first, last; + unit_span(&L, cell.x, cell.y, right_half, t->select_unit, &first, &last); + if (sel_point_cmp(first, t->sel_anchor_first) > 0) first = t->sel_anchor_first; + if (sel_point_cmp(last, t->sel_anchor_last) < 0) last = t->sel_anchor_last; + if (sel_point_cmp(first, last) == 0) + t->selection = false; + else + select_range(t, &L, first, sel_point_prev(&L, last)); + lines_free(&L); + term_write_end(t); +} + +bool term_has_selection(const struct term *t) +{ + if (!t) + return false; + term_write_begin((struct term *)t); + bool has = t->selection && t->sel_end.line_id >= first_retained_line(t); + term_write_end((struct term *)t); + return has; +} + +void term_selection_view_spans(const struct term *t, int *c0, int *c1) +{ + int rows = t->rows, cols = t->cols; + for (int y = 0; y < rows; y++) { + c0[y] = cols; + c1[y] = -1; + } + if (!t->selection) + return; + struct lines L; + lines_init(&L, t); + struct term_sel_point start, end; + if (!selection_bounds(&L, &start, &end)) { + lines_free(&L); + return; + } + uint64_t prev_id = UINT64_MAX; + int prev_seg = -1; + uint32_t prev_end = 0; + + for (int y = 0; y < rows; y++) { + uint64_t id; + int seg; + if (!view_row_line(&L, y, &id, &seg) || id < start.line_id || id > end.line_id) + continue; + uint32_t seg_start; + if (id == prev_id && seg == prev_seg + 1) + seg_start = prev_end; + else + seg_start = line_seg_start_off(&L, id, seg); + + bool wrapped = false; + const struct cell *cells = line_seg_cells(&L, id, seg, NULL, &wrapped); + uint32_t seg_cols = (uint32_t)seg_display_cols(cols, cells, wrapped); + uint32_t seg_end = seg_start + seg_cols; + prev_id = id; + prev_seg = seg; + prev_end = seg_end; + + uint32_t s = 0, e = (uint32_t)(cols - 1); + if (id == start.line_id) { + if (start.cell_off >= seg_end) + continue; + s = start.cell_off > seg_start ? start.cell_off - seg_start : 0; + } + if (id == end.line_id) { + if (end.cell_off < seg_start) + continue; + if (end.cell_off < seg_end && end.cell_off - seg_start < (uint32_t)cols) + e = end.cell_off - seg_start; + } + c0[y] = (int)s; + c1[y] = (int)e; + } + lines_free(&L); +} + +static bool text_reserve(char **text, size_t *cap, size_t need) +{ + if (need <= *cap) + return true; + size_t next = *cap * 2 > need ? *cap * 2 : need; + char *grown = realloc(*text, next); + if (!grown) { + free(*text); + *text = NULL; + return false; + } + *text = grown; + *cap = next; + return true; +} + +static char *range_text_inner(const struct term *t, + const struct term_sel_point *bounds) +{ + if (!t || (!bounds && !t->selection)) return NULL; + struct lines L; + lines_init(&L, t); + struct term_sel_point start, end; + if (bounds) { + start = bounds[0]; + end = bounds[1]; + } else if (!selection_bounds(&L, &start, &end)) { + lines_free(&L); + return NULL; + } + + size_t cap = 4096, n = 0; + char *text = malloc(cap); + for (uint64_t id = start.line_id; text && id <= end.line_id; id++) { + int nsegs = line_total_segs(&L, id); + int start_seg = 0; + if (id == start.line_id) { + if (!line_locate_cell(&L, id, start.cell_off, &start_seg, NULL)) + start_seg = 0; + } + int end_seg = nsegs - 1; + if (id == end.line_id) { + if (!line_locate_cell(&L, id, end.cell_off, &end_seg, NULL)) + end_seg = nsegs - 1; + } + uint32_t next_start = line_seg_start_off(&L, id, start_seg); + int hrows = hist_segs(&L, id); + enum { BATCH_SEGS = 64 }; + struct cell *batch_cells = NULL; + int batch_lens[BATCH_SEGS]; + int batch_start = -1, batch_len = 0; + + for (int seg = start_seg; text && seg <= end_seg && seg < nsegs; seg++) { + uint32_t seg_start = next_start; + bool row_wrapped = false; + const struct cell *cells = NULL; + if (seg < hrows) { + if (seg >= batch_start + batch_len) { + batch_start = seg; + int take = hrows - seg; + if (take > BATCH_SEGS) + take = BATCH_SEGS; + if (seg + take - 1 > end_seg) + take = end_seg - seg + 1; + if (!batch_cells) + batch_cells = malloc((size_t)BATCH_SEGS * (size_t)t->cols * sizeof(struct cell)); + batch_len = batch_cells ? term_hist_decode_segments(&t->hist, id, batch_start, take, + t->cols, batch_cells, batch_lens, + t->gstore) : 0; + } + if (batch_cells && seg >= batch_start && seg < batch_start + batch_len) { + int idx = seg - batch_start; + struct cell *row_buf = batch_cells + (size_t)idx * (size_t)t->cols; + int got = batch_lens[idx]; + row_wrapped = seg + 1 < hrows || is_open_line(&L, id); + if (got < t->cols) { + fill_cells(row_buf + got, t->cols - got, TERM_COLOR_FG, TERM_COLOR_BG); + if (row_wrapped) + for (int x = got; x < t->cols; x++) + row_buf[x].flags |= CELL_WRAP_PAD; + } + cells = row_buf; + } + } + if (!cells) + cells = line_seg_cells(&L, id, seg, NULL, &row_wrapped); + if (!cells) { + next_start = seg_start + (uint32_t)t->cols; + continue; + } + uint32_t seg_end = seg_start + (uint32_t)seg_display_cols(t->cols, cells, row_wrapped); + next_start = seg_end; + + uint32_t s = 0, e = (uint32_t)(t->cols - 1); + if (id == start.line_id) { + if (start.cell_off >= seg_end) + continue; + s = start.cell_off > seg_start ? start.cell_off - seg_start : 0; + } + if (id == end.line_id) { + if (end.cell_off < seg_start) + continue; + if (end.cell_off < seg_end && end.cell_off - seg_start < (uint32_t)t->cols) + e = end.cell_off - seg_start; + } + int c0 = (int)s, c1 = (int)e; + if (c1 < c0) + continue; + bool is_last = id == end.line_id && (end.cell_off < seg_end || seg == nsegs - 1); + bool wrapped = !is_last && row_wrapped; + if (!wrapped) + while (c1 >= c0 && cells[c1].c <= ' ') + c1--; + + if (!text_reserve(&text, &cap, n + (size_t)(c1 - c0 + 1) + 1)) + break; + int c = c0; + for (; c + 3 <= c1; c += 4) { +#if defined(__aarch64__) + uint32x4x4_t block = vld4q_u32((const uint32_t *)&cells[c]); + uint32x4_t cp = block.val[0]; + uint32x4_t valid = vandq_u32( + vcltq_u32(vsubq_u32(cp, vdupq_n_u32(1)), vdupq_n_u32(127)), + vceqq_u32(vandq_u32(block.val[3], + vdupq_n_u32(CELL_TRAIL | CELL_WRAP_PAD)), vdupq_n_u32(0))); + if (!vminvq_u32(valid)) break; + uint16x4_t half = vmovn_u32(cp); + uint8x8_t bytes = vmovn_u16(vcombine_u16(half, half)); + uint32_t packed = vget_lane_u32(vreinterpret_u32_u8(bytes), 0); +#else + uint32_t a = cells[c].c, b = cells[c + 1].c; + uint32_t d = cells[c + 2].c, f = cells[c + 3].c; + unsigned flags = (unsigned)cells[c].flags | cells[c + 1].flags | + cells[c + 2].flags | cells[c + 3].flags; + if ((a - 1) >= 127 || (b - 1) >= 127 || (d - 1) >= 127 || (f - 1) >= 127 || + (flags & (CELL_TRAIL | CELL_WRAP_PAD))) break; + uint32_t packed = a | (b << 8) | (d << 16) | (f << 24); +#endif + memcpy(text + n, &packed, 4); + n += 4; + } + for (; c <= c1; c++) { + uint32_t cp = cells[c].c; + if (cells[c].flags & (CELL_TRAIL | CELL_WRAP_PAD)) + continue; + /* One cell expands to GRAPHEME_MAX_LEN code points of up + * to 4 bytes each. Leave room for all of them. */ + if (!text_reserve(&text, &cap, n + (size_t)GRAPHEME_MAX_LEN * 4 + 1)) + break; + if (cp > 0 && cp < 128) { + text[n++] = (char)cp; + } else { + uint32_t cps[GRAPHEME_MAX_LEN]; + int len = term_gstore_expand(t->gstore, cp ? cp : ' ', cps, GRAPHEME_MAX_LEN); + for (int k = 0; k < len; k++) + n += (size_t)utf8_encode(cps[k], text + n); + } + } + if (text && !is_last && !wrapped && text_reserve(&text, &cap, n + 2)) + text[n++] = '\n'; + } + free(batch_cells); + } + if (text) + text[n] = 0; + lines_free(&L); + return text; +} + +static char *history_text(struct term *t, int max_rows) +{ + if (!t) + return NULL; + term_write_begin(t); + term_hist_prepare_rows(&t->hist, t->cols); + if (!term_hist_view_sync(&t->view, &t->hist, t->cols)) { + term_write_end(t); + return NULL; + } + int end_row = term_screen_history_rows(t) - term_view_offset(t); + if (max_rows <= 0 || end_row <= 0) { + term_write_end(t); + return term_strdup(""); + } + int begin_row = end_row > max_rows ? end_row - max_rows : 0; + struct lines L; + lines_init(&L, t); + struct term_sel_point bounds[2] = {0}; + int first_seg, last_seg; + char *text = NULL; + if (term_hist_view_find_row(&t->view, &t->hist, begin_row, t->cols, + &bounds[0].line_id, &first_seg) && + term_hist_view_find_row(&t->view, &t->hist, end_row - 1, t->cols, + &bounds[1].line_id, &last_seg)) { + bounds[0].cell_off = line_seg_start_off(&L, bounds[0].line_id, first_seg); + uint32_t last_start = line_seg_start_off(&L, bounds[1].line_id, last_seg); + bool wrapped = false; + const struct cell *cells = line_seg_cells(&L, bounds[1].line_id, + last_seg, NULL, &wrapped); + bounds[1].cell_off = last_start + + (uint32_t)seg_display_cols(t->cols, cells, wrapped) - 1; + text = range_text_inner(t, bounds); + if (text && !wrapped) { + size_t n = strlen(text); + size_t cap = n + 1; + if (text_reserve(&text, &cap, n + 2)) { + text[n] = '\n'; + text[n + 1] = '\0'; + } + } + } + lines_free(&L); + term_write_end(t); + return text; +} + +size_t term_history_text(struct term *t, int max_rows, char *buf, size_t cap) +{ + return text_out(history_text(t, max_rows), buf, cap); +} + +size_t term_selection_text(const struct term *t, char *buf, size_t cap) +{ + if (!t) + return 0; + term_write_begin((struct term *)t); + char *text = range_text_inner(t, NULL); + term_write_end((struct term *)t); + return text_out(text, buf, cap); +} diff --git a/src/simd.h b/src/simd.h new file mode 100644 index 0000000..0238500 --- /dev/null +++ b/src/simd.h @@ -0,0 +1,117 @@ +#ifndef TERM_SIMD_H +#define TERM_SIMD_H + +#include "compat.h" + +#include <stdalign.h> +#include <stdbool.h> +#include <stddef.h> +#include <stdint.h> + +#if defined(__x86_64__) || defined(_M_X64) +#include <immintrin.h> +#endif +#if defined(__aarch64__) +#include <arm_neon.h> +#endif + +/* + * x86-64 kernels come in three levels: base (SSE2), AVX2 (x86-64-v3), and + * AVX-512 (x86-64-v4 with VBMI and VBMI2). GCC and Clang compile every + * level with target attributes, and term_isa_init() picks one at run time. + * Other compilers compile the levels that their flags enable. + * + * A kernel at a level is a function marked TERM_TARGET_AVX2 or + * TERM_TARGET_AVX512. It exists when TERM_KERNEL_AVX2 or TERM_KERNEL_AVX512 + * is defined. Call it only when term_use_avx2() or term_use_avx512() is + * true. When the compiler flags already enable a level, these checks are + * constants and the kernels inline into their callers. + */ +#if defined(__AVX512F__) && defined(__AVX512BW__) && defined(__AVX512CD__) && \ + defined(__AVX512DQ__) && defined(__AVX512VL__) && defined(__AVX512VBMI__) && \ + defined(__AVX512VBMI2__) && defined(__AVX2__) && defined(__BMI2__) +#define TERM_ISA_FLOOR 2 +#elif defined(__AVX2__) && defined(__BMI__) && defined(__BMI2__) && defined(__FMA__) && \ + defined(__LZCNT__) +#define TERM_ISA_FLOOR 1 +#else +#define TERM_ISA_FLOOR 0 +#endif + +/* Clang 18 to 21 also need evex512 for 512-bit vectors. GCC and Clang 22 + * reject the name. Apple Clang 17 is LLVM 19. */ +#if defined(__clang__) && ((defined(__apple_build_version__) && __clang_major__ >= 17) || \ + (!defined(__apple_build_version__) && __clang_major__ >= 18 && __clang_major__ <= 21)) +#define TERM_EVEX512 ",evex512" +#else +#define TERM_EVEX512 "" +#endif + +#if (defined(__x86_64__) || defined(_M_X64)) && defined(TERM_COMPILER_GNU) +#define TERM_TARGET_AVX2 \ + __attribute__((target("avx2,bmi,bmi2,fma,lzcnt,popcnt,movbe,f16c"))) +#define TERM_TARGET_AVX512 \ + __attribute__((target("avx2,bmi,bmi2,fma,lzcnt,popcnt,movbe,f16c,avx512f,avx512bw," \ + "avx512cd,avx512dq,avx512vl,avx512vbmi,avx512vbmi2" TERM_EVEX512))) +#define TERM_KERNEL_AVX2 1 +#define TERM_KERNEL_AVX512 1 +#define TERM_ISA_RUNTIME 1 +#else +#define TERM_TARGET_AVX2 +#define TERM_TARGET_AVX512 +#if TERM_ISA_FLOOR >= 1 +#define TERM_KERNEL_AVX2 1 +#endif +#if TERM_ISA_FLOOR >= 2 +#define TERM_KERNEL_AVX512 1 +#endif +#endif + +enum { TERM_ISA_BASE, TERM_ISA_AVX2, TERM_ISA_AVX512 }; + +/* The level in use. Zero, the base level, until term_isa_init() runs. It + * is a plain int, so the compiler can hoist the checks out of loops: it is + * written once, before the release that publishes it, and read only after + * that. */ +extern int term_isa_level; + +/* Pick the level once. Any thread can call it. term_new() calls it before + * the first terminal exists. */ +void term_isa_init(void); + +static inline bool term_use_avx2(void) +{ +#if TERM_ISA_FLOOR >= 1 + return true; +#elif defined(TERM_ISA_RUNTIME) + return term_isa_level >= TERM_ISA_AVX2; +#else + return false; +#endif +} + +static inline bool term_use_avx512(void) +{ +#if TERM_ISA_FLOOR >= 2 + return true; +#elif defined(TERM_ISA_RUNTIME) + return term_isa_level >= TERM_ISA_AVX512; +#else + return false; +#endif +} + +#if defined(__aarch64__) +static inline uint16_t term_lane_bits(uint8x16_t m) +{ + const uint8x16_t idx = {1, 2, 4, 8, 16, 32, 64, 128, + 1, 2, 4, 8, 16, 32, 64, 128}; + uint8x16_t s = vandq_u8(m, idx); + s = vpaddq_u8(s, s); + s = vpaddq_u8(s, s); + s = vpaddq_u8(s, s); + return (uint16_t)(vgetq_lane_u8(s, 0) | (vgetq_lane_u8(s, 1) << 8)); +} +#endif + +#endif diff --git a/src/snapshot.c b/src/snapshot.c new file mode 100644 index 0000000..dcd1dc0 --- /dev/null +++ b/src/snapshot.c @@ -0,0 +1,346 @@ +#include "compat.h" +#include "screen.h" +#include "unicode/grapheme.h" +#include <stdlib.h> +#include <string.h> + +/* + * A snapshot keeps one converted copy of each viewport row. An update + * copies a row again only when the terminal reports a new key for it. + * Cluster cells refer to an arena of [length, code points...] records that + * the snapshot owns, so a renderer never reads the terminal's cluster table. + */ +struct term_snapshot { + struct term_view view; + uint32_t palette[TERM_PALETTE_SIZE]; + uint64_t source; /* serial of the terminal it copied */ + + struct term_cell *cells; /* rows * cols */ + uint64_t *keys; /* per row */ + uint32_t *row_clusters; /* arena records each row holds */ + struct term_span *selection; /* per row */ + size_t cell_cap; + int row_cap; + + uint32_t *arena; + uint32_t arena_len, arena_cap, arena_dead; +}; + +struct term_snapshot *term_snapshot_new(void) +{ + struct term_snapshot *s = calloc(1, sizeof(*s)); + if (s) + s->view.palette = s->palette; + return s; +} + +void term_snapshot_free(struct term_snapshot *s) +{ + if (!s) + return; + free(s->cells); + free(s->keys); + free(s->row_clusters); + free(s->selection); + free(s->arena); + free(s); +} + +static bool reserve(struct term_snapshot *s, int cols, int rows) +{ + size_t cells = (size_t)cols * (size_t)rows; + if (cells > s->cell_cap) { + struct term_cell *c = realloc(s->cells, cells * sizeof(*c)); + if (!c) + return false; + s->cells = c; + s->cell_cap = cells; + } + if (rows > s->row_cap) { + uint64_t *k = realloc(s->keys, (size_t)rows * sizeof(*k)); + if (k) + s->keys = k; + uint32_t *rc = realloc(s->row_clusters, (size_t)rows * sizeof(*rc)); + if (rc) + s->row_clusters = rc; + struct term_span *sel = realloc(s->selection, (size_t)rows * sizeof(*sel)); + if (sel) + s->selection = sel; + if (!k || !rc || !sel) + return false; + s->row_cap = rows; + } + return true; +} + +/* Attribute contributions of the low and high byte of cell flags. */ +static uint16_t lo_attrs[256], hi_attrs[256]; +static uint8_t hi_underline[256]; +void term_snapshot_tables_init(void) +{ + for (int b = 0; b < 256; b++) { + uint16_t lo = 0, hi = 0; + if (b & CELL_BOLD) lo |= TERM_ATTR_BOLD; + if (b & CELL_ITALIC) lo |= TERM_ATTR_ITALIC; + if (b & CELL_STRIKE) lo |= TERM_ATTR_STRIKE; + if (b & CELL_OVERLINE) lo |= TERM_ATTR_OVERLINE; + if (b & CELL_EMOJI) lo |= TERM_ATTR_EMOJI; + if (b & CELL_TEXT) lo |= TERM_ATTR_TEXT; + uint16_t f = (uint16_t)(b << 8); + if (f & CELL_FAINT) hi |= TERM_ATTR_FAINT; + if (f & CELL_LINK) hi |= TERM_ATTR_LINK; + uint8_t ul = 0; + if (f & CELL_DOUBLE_UNDERLINE) ul = TERM_UNDERLINE_DOUBLE; + else if (f & CELL_CURLY_UNDERLINE) ul = TERM_UNDERLINE_CURLY; + else if (f & CELL_DOTTED_UNDERLINE) ul = TERM_UNDERLINE_DOTTED; + else if (f & CELL_DASHED_UNDERLINE) ul = TERM_UNDERLINE_DASHED; + lo_attrs[b] = lo; + hi_attrs[b] = hi; + hi_underline[b] = ul; + } +} + +static uint32_t arena_add(struct term_snapshot *s, const struct gstore *g, uint32_t handle) +{ + uint32_t cps[GRAPHEME_MAX_LEN]; + int n = term_gstore_expand(g, handle, cps, GRAPHEME_MAX_LEN); + if (n <= 0) + return ' '; + if (s->arena_len + (uint32_t)n + 1 > s->arena_cap) { + uint32_t cap = s->arena_cap ? s->arena_cap * 2 : 256; + while (cap < s->arena_len + (uint32_t)n + 1) + cap *= 2; + uint32_t *a = realloc(s->arena, cap * sizeof(*a)); + if (!a) + return cps[0]; + s->arena = a; + s->arena_cap = cap; + } + uint32_t off = s->arena_len; + s->arena[off] = (uint32_t)n; + memcpy(&s->arena[off + 1], cps, (size_t)n * sizeof(*cps)); + s->arena_len += (uint32_t)n + 1; + return TERM_TEXT_CLUSTER | off; +} + +/* Convert one stored row; returns the number of arena words it added. */ +static uint32_t convert_row(struct term_snapshot *s, const struct gstore *g, + struct term_cell *dst, + const struct cell *src, int cols) +{ + uint32_t before = s->arena_len; + for (int x = 0; x < cols; x++) { + const struct cell *c = &src[x]; + uint16_t f = c->flags; + uint8_t lo = (uint8_t)f, hi = (uint8_t)(f >> 8); + uint16_t ul = hi_underline[hi]; + if (!ul && (f & CELL_UNDERLINE)) + ul = TERM_UNDERLINE_SINGLE; + uint32_t text = c->c; + if (TERM_EXPECT(text < 0x20, 0)) + text = text ? ' ' : 0; + else if (TERM_EXPECT(grapheme_is_handle(text), 0)) + text = arena_add(s, g, text); + dst[x] = (struct term_cell){ + .text = text, + .fg = c->fg, + .bg = c->bg, + .attrs = (uint16_t)(lo_attrs[lo] | hi_attrs[hi] | + ul << TERM_ATTR_UNDERLINE_SHIFT), + .width = (f & CELL_TRAIL) ? 0 : (f & CELL_WIDE) ? 2 : 1, + }; + } + return s->arena_len - before; +} + +static bool term_snapshot_update_inner(struct term_snapshot *s, struct term *t) +{ + if (!s || !t) + return false; + int cols = t->cols, rows = t->rows; + if (!reserve(s, cols, rows)) { + s->view.size = (struct term_vector2i){ 0 }; + s->source = 0; + return false; + } + + bool resized = s->view.size.x != cols || s->view.size.y != rows; + bool all = resized || s->source != t->serial; + /* Reclaim cluster records once most of the arena is garbage. */ + if (s->arena_dead > 4096 && s->arena_dead * 2 > s->arena_len) + all = true; + if (all) { + s->arena_len = 0; + s->arena_dead = 0; + } + + bool changed = all; + term_hist_prepare_rows(&t->hist, cols); + if (!term_hist_view_sync(&t->view, &t->hist, cols)) + return false; + term_hist_view_refine(&t->view, &t->hist, cols, rows); + uint64_t stack_keys[256]; + uint64_t *new_keys = rows <= 256 ? stack_keys : malloc((size_t)rows * sizeof(uint64_t)); + if (!new_keys) + return false; + for (int y = 0; y < rows; y++) + new_keys[y] = term_screen_view_row_key(t, y); + + int scrolled = 0; + if (!all && rows > 1) { + int cand_k = 0; + /* Find candidate scroll offset k by checking where top rows moved. */ + for (int p = 0; p < 2 && p < rows - 1 && !cand_k; p++) { + uint64_t key = new_keys[p]; + if (!key) + continue; + for (int k = 1; p + k < rows; k++) { + if (s->keys[p + k] == key) { + cand_k = k; + break; + } + } + } + + if (cand_k > 0) { + int total = rows - cand_k; + int threshold = total / 2; + int matches = 0, mismatches = 0; + for (int y = 0; y < total; y++) { + if (new_keys[y] == s->keys[y + cand_k]) { + if (++matches > threshold) { + scrolled = cand_k; + break; + } + } else { + if (++mismatches > threshold) + break; + } + } + } + } + + if (scrolled > 0) { + int k = scrolled; + memmove(s->cells, s->cells + (size_t)k * cols, + (size_t)(rows - k) * cols * sizeof(struct term_cell)); + memmove(s->row_clusters, s->row_clusters + k, + (size_t)(rows - k) * sizeof(uint32_t)); + memmove(s->keys, s->keys + k, + (size_t)(rows - k) * sizeof(uint64_t)); + for (int y = rows - k; y < rows; y++) { + s->keys[y] = 0; + s->row_clusters[y] = 0; + } + } + + for (int y = 0; y < rows; y++) { + uint64_t key = new_keys[y]; + if (!all && s->keys[y] == key) + continue; + if (!all) + s->arena_dead += s->row_clusters[y]; + const struct cell *src = term_screen_view_row(t, y); + s->row_clusters[y] = convert_row(s, t->gstore, s->cells + (size_t)y * cols, src, cols); + s->keys[y] = key; + changed = true; + } + if (new_keys != stack_keys) + free(new_keys); + + int stack[2][256]; + int *c0 = rows <= 256 ? stack[0] : malloc((size_t)rows * 2 * sizeof(int)); + if (!c0) + return false; + int *c1 = rows <= 256 ? stack[1] : c0 + rows; + term_selection_view_spans(t, c0, c1); + for (int y = 0; y < rows; y++) { + struct term_span span = c1[y] >= c0[y] ? (struct term_span){ c0[y], c1[y] + 1 } + : (struct term_span){ 0 }; + if (all || span.begin != s->selection[y].begin || span.end != s->selection[y].end) + changed = true; + s->selection[y] = span; + } + if (c0 != stack[0]) + free(c0); + + struct term_cursor cursor = term_cursor(t); + int view_offset = term_view_offset(t); + cursor.pos.y += view_offset; + if (cursor.pos.y >= rows) + cursor.visible = false; + const struct term_cursor *old = &s->view.cursor; + if (cursor.pos.x != old->pos.x || cursor.pos.y != old->pos.y || + cursor.style != old->style || cursor.visible != old->visible || + cursor.blink != old->blink || + s->view.scroll_offset != view_offset || + s->view.reverse_video != t->reverse_video) + changed = true; + if (all || s->view.palette_version != t->palette.version) { + memcpy(s->palette, t->palette.colors, sizeof(s->palette)); + s->view.palette_version = t->palette.version; + changed = true; + } + + s->source = t->serial; + s->view.size = (struct term_vector2i){ .x = cols, .y = rows }; + s->view.cursor = cursor; + s->view.scroll_offset = view_offset; + s->view.scrolled = scrolled; + s->view.reverse_video = t->reverse_video; + if (changed || scrolled) + s->view.version++; + return true; +} + +bool term_snapshot_update(struct term_snapshot *s, struct term *t) +{ + if (!s || !t) + return false; + term_write_begin(t); + bool ok = term_snapshot_update_inner(s, t); + term_write_end(t); + return ok; +} + +const struct term_view *term_snapshot_view(const struct term_snapshot *s) +{ + return &s->view; +} + +const struct term_cell *term_snapshot_row(const struct term_snapshot *s, int y) +{ + if (y < 0 || y >= s->view.size.y) + return NULL; + return s->cells + (size_t)y * s->view.size.x; +} + +uint64_t term_snapshot_row_key(const struct term_snapshot *s, int y) +{ + if (y < 0 || y >= s->view.size.y) + return 0; + return s->keys[y]; +} + +struct term_span term_snapshot_row_selection(const struct term_snapshot *s, int y) +{ + if (y < 0 || y >= s->view.size.y) + return (struct term_span){ 0 }; + return s->selection[y]; +} + +int term_snapshot_cluster(const struct term_snapshot *s, uint32_t text, uint32_t *cps, int cap) +{ + if (!(text & TERM_TEXT_CLUSTER)) { + if (cap > 0) + cps[0] = text; + return 1; + } + uint32_t off = text & ~TERM_TEXT_CLUSTER; + if (off >= s->arena_len) + return 0; + int n = (int)s->arena[off]; + for (int i = 0; i < n && i < cap; i++) + cps[i] = s->arena[off + 1 + i]; + return n; +} diff --git a/src/term.c b/src/term.c new file mode 100644 index 0000000..583bdd8 --- /dev/null +++ b/src/term.c @@ -0,0 +1,125 @@ +#include "internal.h" +#include "screen.h" + +_Thread_local char term_thread_anchor; + +const char *term_unicode_version(void) +{ + return UNICODE_DATA_VERSION; +} + +void term_lock(struct term *t) +{ + term_write_begin(t); +} + +void term_unlock(struct term *t) +{ + term_write_end(t); +} + +struct term_vector2i term_size(const struct term *t) +{ + if (!t) + return (struct term_vector2i){ 0 }; + term_write_begin((struct term *)t); + struct term_vector2i size = { .x = t->cols, .y = t->rows }; + term_write_end((struct term *)t); + return size; +} + +struct term_modes term_modes(const struct term *t) +{ + if (!t) + return (struct term_modes){ 0 }; + term_write_begin((struct term *)t); + struct term_modes m = { + .keyboard = t->keyboard[t->alternate].flags, + .mouse = (enum term_mouse)t->mouse_mode, + .mouse_encoding = (enum term_mouse_encoding)t->mouse_encoding, + .grid = { .x = t->cols, .y = t->rows }, + .app_cursor = t->application_cursor, + .app_keypad = t->application_keypad, + .backarrow_bs = t->backarrow_sends_bs, + .bracketed_paste = t->bracketed_paste, + .focus_events = t->focus_events, + .alternate = t->alternate, + .reverse_video = t->reverse_video, + .synchronized = t->synchronized_output, + }; + term_write_end((struct term *)t); + return m; +} + +struct term_cursor term_cursor(const struct term *t) +{ + if (!t) + return (struct term_cursor){ .visible = true }; + term_write_begin((struct term *)t); + struct term_cursor c = { + .pos = { .x = t->cursor_x, .y = t->cursor_y }, + .style = t->cursor_style, + .visible = t->cursor_visible, + .blink = t->cursor_blink, + }; + term_write_end((struct term *)t); + return c; +} + +void term_set_cursor_default(struct term *t, enum term_cursor_style style, bool steady) +{ + if (!t) + return; + term_write_begin(t); + if (style > TERM_CURSOR_BAR) + style = TERM_CURSOR_BLOCK; + if (t->cursor_style != style || t->cursor_blink == steady) { + struct srow *cur = term_row(t, t->cursor_y); + if (cur) cur->version++; + } + t->default_cursor_style = t->cursor_style = style; + t->default_cursor_blink = t->cursor_blink = !steady; + term_write_end(t); +} + +void term_set_theme(struct term *t, const struct term_theme *theme) +{ + if (!t) + return; + term_write_begin(t); + struct palette next = { .version = t->palette.version }; + term_palette_init(&next, theme); + if (memcmp(&next.theme, &t->palette.theme, sizeof(next.theme))) + t->palette = next; + term_write_end(t); +} + +void term_focus(struct term *t, bool focused) +{ + if (!t) + return; + term_write_begin(t); + if (t->focus_events) + send_response(t, focused ? "\033[I" : "\033[O"); + term_write_end(t); +} + +size_t term_cwd(const struct term *t, char *buf, size_t cap) +{ + if (!t) + return term_copy_out(NULL, buf, cap); + term_write_begin((struct term *)t); + size_t n = term_copy_out(t->cwd, buf, cap); + term_write_end((struct term *)t); + return n; +} + +size_t term_pointer_shape(const struct term *t, char *buf, size_t cap) +{ + if (!t) + return term_copy_out(NULL, buf, cap); + term_write_begin((struct term *)t); + size_t n = term_copy_out(t->pointer_shape, buf, cap); + term_write_end((struct term *)t); + return n; +} diff --git a/src/text.c b/src/text.c new file mode 100644 index 0000000..9f73bd3 --- /dev/null +++ b/src/text.c @@ -0,0 +1,350 @@ +#include "compat.h" +#include "core.h" +#include "kernels.h" +#include "text_run.h" +#include "screen.h" +#include "escape.h" +#include "palette.h" +#include "unicode/grapheme.h" +#include "unicode/unicode.h" +#include "text.h" +#include "unicode/utf8.h" +#include <stdio.h> +#include <string.h> + +static void dispatch_c1_control(struct term *t, uint32_t cp) +{ + static const char c1_map[32] = { + [4] = 'D', [5] = 'E', [8] = 'H', [13] = 'M', + [16] = 'P', [27] = '[', [28] = '\\', [29] = ']' }; + char ch = c1_map[cp - 0x80]; + if (!ch) return; + t->state = ST_ESC; + term_escape_byte(t, (unsigned char)ch); +} + +static inline void emit_codepoint(struct term *t, uint32_t cp, int klass) +{ + static const uint32_t dec_graphics[] = { + 0x00a0, + 0x25c6,0x2592,0x2409,0x240c,0x240d,0x240a,0x00b0,0x00b1, + 0x2424,0x240b,0x2518,0x2510,0x250c,0x2514,0x253c,0x23ba, + 0x23bb,0x2500,0x23bc,0x23bd,0x251c,0x2524,0x2534,0x252c, + 0x2502,0x2264,0x2265,0x03c0,0x2260,0x00a3,0x00b7 }; + if (cp >= 0x80 && cp <= 0x9f) { + dispatch_c1_control(t, cp); + return; + } + if (t->graphics && cp >= '_' && cp <= '~') { + cp = dec_graphics[cp - '_']; + klass = -1; + } + term_sc_print_glyph(t, cp, klass, term_current_flags(t)); +} + +static inline void emit_decoded(struct term *t, uint32_t cp) +{ + int klass = term_text_class(cp); + if (!term_sc_extend_cluster(t, cp, klass)) + emit_codepoint(t, cp, klass); +} + +/* A byte that cannot continue a UTF-8 sequence ends it: the unfinished + * sequence prints U+FFFD (Unicode 3.9, maximal subparts). */ +void term_text_utf8_interrupt(struct term *t) +{ + if (TERM_EXPECT(t->utf8_state == UTF8_ACCEPT, 1)) + return; + t->utf8_state = UTF8_ACCEPT; + emit_decoded(t, 0xFFFD); +} + +void term_text_ground(struct term *t, unsigned char ch) +{ + if (ch >= 0x20 && ch < 0x7f && t->utf8_state == UTF8_ACCEPT) { + emit_codepoint(t, ch, 1); + return; + } + + if (ch < 0x20 || ch == 0x7f) + term_text_utf8_interrupt(t); + + if (TERM_EXPECT(ch == 0x1b, 0)) { + term_escape_begin(t); + } else if (ch == '\n' || ch == 0x0b || ch == 0x0c) { + /* VT and FF act as LF (xterm). */ + if (term_lnm(t)) sc_cr(t); + sc_linefeed(t, 1); + } else if (ch == '\r') { + sc_cr(t); + } else if (ch == '\b') { + sc_bs(t); + } else if (ch == '\t') { + term_sc_tab(t, 1); + } else if (ch == 0x07) { + if (t->cb.bell) + t->cb.bell(t->cb.ud); + } else if (ch == 0x0e) { + t->charset_active = 1; + t->graphics = t->charsets[1] == '0'; + } else if (ch == 0x0f) { + t->charset_active = 0; + t->graphics = t->charsets[0] == '0'; + } else if (ch >= 0x20 && ch != 0x7f) { + uint32_t before = t->utf8_state; + uint32_t st = utf8_decode(&t->utf8_state, &t->utf8_codep, ch); + if (st == UTF8_ACCEPT) { + emit_decoded(t, t->utf8_codep); + } else if (st == UTF8_REJECT) { + t->utf8_state = UTF8_ACCEPT; + emit_decoded(t, 0xFFFD); + /* The byte that ended an unfinished sequence starts anew. */ + if (before != UTF8_ACCEPT) + term_text_ground(t, ch); + } + } +} + +static inline int decode_ground_utf8(struct term *t, const char *buf, size_t len, + uint32_t *cp) +{ + return utf8_decode_ground(&t->utf8_state, &t->utf8_codep, buf, len, cp); +} + +static inline bool can_stream_bottom(const struct term *t) +{ + return t->state == ST_GROUND && !t->graphics && !t->insert_mode && t->autowrap && + !t->lr_margin_mode && + t->utf8_state == UTF8_ACCEPT && !t->inverse && + t->scroll_top == 0 && t->scroll_bottom == t->rows - 1 && + t->cursor_y == t->rows - 1 && + !t->alternate && t->sb_capacity > 0 && + using_default_colors(t); +} + +static inline bool try_text_run(struct term *t, const char *buf, size_t len, + size_t *idx) +{ + size_t i = *idx; + if (t->pending_wrap || i + 4 > len || + ((unsigned char)buf[i] >= 0x80 && + ((unsigned char)buf[i] < 0xc2 || (unsigned char)buf[i] > 0xf4)) || + t->last_char == 0x200D || grapheme_is_handle(t->last_char)) + return false; + if ((unsigned char)buf[i] >= 0x80) { + uint32_t cp; + int bytes = utf8_decode_one(buf + i, len - i, &cp); + if (!bytes || term_text_class(cp) <= 0) + return false; + } + if (t->cursor_x != 0 && + t->cursor_x != term_row(t, t->cursor_y)->len) + return false; + uint32_t fg = t->inverse ? t->cur_bg : t->cur_fg; + uint32_t bg = t->inverse ? t->cur_fg : t->cur_bg; + uint32_t flags = term_flags_pad(t); + bool wrote = false; + do { + struct srow *text = t->insert_mode || t->lr_margin_mode ? NULL : text_append_row(t); + if (text) { + int x = t->cursor_x, avail = t->cols - x; + struct text_run_bytes r = term_text_run_bytes(text->text.text + text->text.text_len, + buf + i, (int)(len - i), avail, + t->last_char, &t->text_width); + if (!r.bytes) + break; + wrote = true; + text_commit(text, r.out, r.chars, r.cols, + (uint8_t)((r.narrow ? HLINE_NARROW : 0) | (r.wide ? HLINE_HAS_WIDE : 0)), + fg, bg, flags); + t->last_char = r.last; + sc_text_advance(t, x, r.cols, avail); + i += (size_t)r.bytes; + } else { + struct span span = sc_span_begin(t, t->cols); + if (span.avail < 2) + break; + struct text_run_result r = term_text_run(span.cells, buf + i, (int)(len - i), + span.avail, fg, bg, flags, t->last_char, + &t->text_width); + if (!r.bytes) + break; + wrote = true; + sc_span_end(t, &span, r.cols, 0, r.last); + i += (size_t)r.bytes; + } + bool newline = false; + if (i < len && buf[i] == '\r') { + sc_cr(t); + i++; + if (i < len && buf[i] == '\n') { + sc_linefeed(t, 1); + i++; + newline = true; + } + } else if (i < len && buf[i] == '\n') { + if (term_lnm(t)) sc_cr(t); + sc_linefeed(t, 1); + i++; + newline = true; + } + if (!newline || i + 4 > len || t->pending_wrap || + t->last_char == 0x200D || grapheme_is_handle(t->last_char) || + (t->cursor_x != 0 && + t->cursor_x != term_row(t, t->cursor_y)->len)) + break; + } while (true); + if (wrote) *idx = i - 1; + return wrote; +} + +static void term_feed_inner(struct term *t, const char *buf, size_t len, unsigned flags) +{ + if (!t || !buf || len == 0) + return; + t->transport_newline = (flags & TERM_FEED_ONLCR) != 0; + + for (size_t i = 0; i < len; i++) { + while (t->state != ST_GROUND) { + if (t->state == ST_CSI) + i += term_escape_csi(t, buf + i, len - i); + else if (t->state == ST_OSC || t->state == ST_DCS || t->state == ST_IGNORE) + i += term_escape_str(t, buf + i, len - i); + else + term_escape_byte(t, (unsigned char)buf[i++]); + if (i == len) return; + } + + unsigned char ch = (unsigned char)buf[i]; + if ((ch < 0x20 || ch == 0x7f) && t->state == ST_GROUND) + term_text_utf8_interrupt(t); + + if (TERM_EXPECT(ch < 0x20 && t->state == ST_GROUND && + ch != 0x1b && ch != '\n' && ch != '\r' && ch != '\b' && ch != '\t' && + ch != 0x0e && ch != 0x0f && ch != 0x07 && ch != 0x0b && + ch != 0x0c, 0)) { + continue; + } + + if (TERM_EXPECT(ch == 0x1b && t->state == ST_GROUND && + t->utf8_state == UTF8_ACCEPT && i + 1 < len && + buf[i + 1] == '[' && !t->insert_mode && + !t->lr_margin_mode && !t->graphics, 0)) { + int fast = term_try_fast_csi(t, buf + i + 2, len - i - 2); + if (fast > 0) { + i += 1 + (size_t)fast; + continue; + } + t->state = ST_CSI; + csi_parser_reset(&t->csi_parser); + i += 1 + term_escape_csi(t, buf + i + 2, len - i - 2); + continue; + } + + if (TERM_EXPECT(t->state == ST_GROUND && ch == '\n', 0)) { + size_t end = i + 1 + scan_byte_run(buf + i + 1, len - i - 1, '\n'); + if (term_lnm(t)) sc_cr(t); + sc_linefeed(t, end - i); + i = end - 1; + continue; + } + + if (TERM_EXPECT(t->state == ST_GROUND && ch == '\r' && + i + 1 < len && buf[i + 1] == '\n', 0)) { + size_t end = i + 2 + scan_crlf_pairs(buf + i + 2, len - i - 2); + sc_cr(t); + sc_linefeed(t, (end - i) / 2); + i = end - 1; + continue; + } + + if (TERM_EXPECT(ch >= 0x20 && ch < 0x7f && !t->insert_mode && !t->lr_margin_mode && + t->state == ST_GROUND && !t->graphics && + t->utf8_state == UTF8_ACCEPT, 1)) { + int avail = t->pending_wrap ? t->cols : t->cols - t->cursor_x; + int rem = (int)(len - i); + if (rem > avail) rem = avail; + int run = 1; + if (rem > 1 && (unsigned char)buf[i + 1] >= 0x20 && + (unsigned char)buf[i + 1] < 0x7f) + run += scan_ascii_run(buf + i + 1, rem - 1); + if (TERM_EXPECT(can_stream_bottom(t), 1) && + (run == t->cols || + (i + (size_t)run < len && + (buf[i + run] == '\r' || buf[i + run] == '\n')))) { + struct srow *row = term_row(t, t->cursor_y); + if (t->pending_wrap || + (t->cursor_x == 0 && row && row->len == 0)) { + size_t done = term_sc_bulk_ascii_lines(t, buf + i, len - i); + if (done > 0) { + i += done - 1; + continue; + } + } + } + if (i + (size_t)run < len && (unsigned char)buf[i + run] >= 0x80) { + uint32_t next_cp; + if (utf8_decode_one(buf + i + run, len - i - (size_t)run, &next_cp) && + term_text_class(next_cp) > 0 && + try_text_run(t, buf, len, &i)) + continue; + } + sc_print_ascii(t, buf + i, run); + i += (size_t)run; + if (i < len && buf[i] == '\r') { + sc_cr(t); + i++; + if (i < len && buf[i] == '\n') { + sc_linefeed(t, 1); + i++; + } + } else if (i < len && buf[i] == '\n') { + if (term_lnm(t)) sc_cr(t); + sc_linefeed(t, 1); + i++; + } + i--; + continue; + } + + if (TERM_EXPECT(ch == '\r' && t->state == ST_GROUND, 0)) { + sc_cr(t); + continue; + } + + if (TERM_EXPECT(ch >= 0x80 && t->state == ST_GROUND && + t->utf8_state == UTF8_ACCEPT && !t->graphics, 1)) { + if (!t->insert_mode && !t->lr_margin_mode && + try_text_run(t, buf, len, &i)) + continue; + uint32_t cp; + int cp_len = decode_ground_utf8(t, buf + i, len - i, &cp); + if (cp) + emit_decoded(t, cp); + i += (size_t)(cp_len - 1); + continue; + } + + if (TERM_EXPECT(t->state == ST_GROUND, 1)) + term_text_ground(t, ch); + else + term_escape_byte(t, ch); + } +} + +void term_feed(struct term *t, const char *buf, size_t len, unsigned flags) +{ + if (!t || !buf || len == 0) + return; + term_write_begin(t); + term_feed_inner(t, buf, len, flags); + term_write_end(t); +} + +void term_text_repeat(struct term *t, int count) +{ + int screen = t->cols * t->rows; + if (count > screen) count = screen; + for (int j = 0; j < count && t->last_char; j++) + term_sc_print_glyph(t, t->last_char, -1, term_current_flags(t)); +} diff --git a/src/text.h b/src/text.h new file mode 100644 index 0000000..9caaf80 --- /dev/null +++ b/src/text.h @@ -0,0 +1,10 @@ +#ifndef TERM_TEXT_H +#define TERM_TEXT_H +#include "internal.h" + +void term_text_ground(struct term *t, unsigned char ch); +/* End an unfinished UTF-8 sequence: it prints U+FFFD. */ +void term_text_utf8_interrupt(struct term *t); +void term_text_repeat(struct term *t, int count); + +#endif diff --git a/src/text_run.c b/src/text_run.c new file mode 100644 index 0000000..3901cbe --- /dev/null +++ b/src/text_run.c @@ -0,0 +1,682 @@ +#include <assert.h> +#include <stdalign.h> +#include "text_run.h" +#include "compat.h" +#include "kernels.h" +#include "simd.h" +#include "unicode/utf8.h" + +static uint8_t unicode_fast_class[65536]; +static int8_t unicode_width_bmp[65536]; + +/* Runs of code points with one fast class, for the AVX-512 width cache. */ +#define TERM_WIDTH_RUNS_MAX 1024 +static uint16_t width_run_lo[TERM_WIDTH_RUNS_MAX]; +static uint16_t width_run_hi[TERM_WIDTH_RUNS_MAX]; +static uint8_t width_run_class[TERM_WIDTH_RUNS_MAX]; +static int width_run_count; + +void term_text_tables_init(void) +{ + for (uint32_t cp = 0; cp < 65536; cp++) { + int w = unicode_width(cp); + unicode_fast_class[cp] = (w == 1 || w == 2) && !cluster_contextual(cp) + ? (uint8_t)w : 0; + unicode_width_bmp[cp] = (int8_t)w; + } + for (uint32_t cp = 0; cp < 65536; ) { + uint8_t k = unicode_fast_class[cp]; + uint32_t lo = cp; + while (cp < 65536 && unicode_fast_class[cp] == k) cp++; + if (width_run_count == TERM_WIDTH_RUNS_MAX) { + width_run_count = 0; + break; + } + width_run_lo[width_run_count] = (uint16_t)lo; + width_run_hi[width_run_count] = (uint16_t)(cp - 1); + width_run_class[width_run_count] = k; + width_run_count++; + } +} + +static TERM_MAYBE_UNUSED int width_run_find(uint32_t cp) +{ + int lo = 0, hi = width_run_count - 1; + while (lo < hi) { + int mid = lo + (hi - lo + 1) / 2; + if (width_run_lo[mid] > cp) hi = mid - 1; + else lo = mid; + } + return width_run_count > 0 ? lo : -1; +} + +static TERM_MAYBE_UNUSED uint16_t width_cache_covers(const struct width_cache *wc, + const uint32_t *cp) +{ + uint16_t covered = 0; + for (int j = 0; j < TERM_WIDTH_SLOTS; j++) + for (int l = 0; l < 16; l++) + if (cp[l] - wc->lo[j] <= wc->span[j]) + covered |= (uint16_t)(1u << l); + return covered; +} + +static TERM_MAYBE_UNUSED bool width_cache_extend(struct width_cache *wc, const uint32_t *cp, + uint16_t covered) +{ + unsigned miss = (unsigned)~covered & 0xFFFFu; + int r = width_run_find(cp[miss ? term_ctz32(miss) : 0]); + if (r < 0) return false; + uint8_t k = width_run_class[r]; + if (k != 1 && k != 2) return false; + if (k != wc->klass) { + for (int j = 0; j < TERM_WIDTH_SLOTS; j++) { + wc->lo[j] = 0xFFFFFFFFu; + wc->span[j] = 0; + } + wc->klass = k; + wc->next = 0; + } + for (int added = 0; added < TERM_WIDTH_SLOTS; added++) { + wc->lo[wc->next] = width_run_lo[r]; + wc->span[wc->next] = (uint32_t)(width_run_hi[r] - width_run_lo[r]); + wc->next = (uint8_t)((wc->next + 1) % TERM_WIDTH_SLOTS); + covered = width_cache_covers(wc, cp); + if (covered == 0xFFFF) return true; + r = width_run_find(cp[term_ctz32((unsigned)~covered & 0xFFFFu)]); + if (r < 0 || width_run_class[r] != k) return false; + } + return false; +} + +int term_text_class(uint32_t cp) +{ + assert(term_tables_ready()); + return TERM_EXPECT(cp < 65536, 1) ? (int)unicode_width_bmp[cp] : unicode_width(cp); +} + +#if defined(TERM_KERNEL_AVX512) + +static const uint8_t utf8_3byte_shuffle[64] = { + 2, 1, 0, 48, 5, 4, 3, 48, 8, 7, 6, 48, 11, 10, 9, 48, + 14, 13, 12, 48, 17, 16, 15, 48, 20, 19, 18, 48, 23, 22, 21, 48, + 26, 25, 24, 48, 29, 28, 27, 48, 32, 31, 30, 48, 35, 34, 33, 48, + 38, 37, 36, 48, 41, 40, 39, 48, 44, 43, 42, 48, 47, 46, 45, 48, +}; + +static const uint8_t utf8_3byte_bits[64] = { + 0xF0, 0xC0, 0xC0, 0xF0, 0xC0, 0xC0, 0xF0, 0xC0, 0xC0, 0xF0, 0xC0, 0xC0, + 0xF0, 0xC0, 0xC0, 0xF0, 0xC0, 0xC0, 0xF0, 0xC0, 0xC0, 0xF0, 0xC0, 0xC0, + 0xF0, 0xC0, 0xC0, 0xF0, 0xC0, 0xC0, 0xF0, 0xC0, 0xC0, 0xF0, 0xC0, 0xC0, + 0xF0, 0xC0, 0xC0, 0xF0, 0xC0, 0xC0, 0xF0, 0xC0, 0xC0, 0xF0, 0xC0, 0xC0, + 0xF0, 0xC0, 0xC0, 0xF0, 0xC0, 0xC0, 0xF0, 0xC0, 0xC0, 0xF0, 0xC0, 0xC0, + 0xF0, 0xC0, 0xC0, 0xF0, +}; +static const uint8_t utf8_3byte_want[64] = { + 0xE0, 0x80, 0x80, 0xE0, 0x80, 0x80, 0xE0, 0x80, 0x80, 0xE0, 0x80, 0x80, + 0xE0, 0x80, 0x80, 0xE0, 0x80, 0x80, 0xE0, 0x80, 0x80, 0xE0, 0x80, 0x80, + 0xE0, 0x80, 0x80, 0xE0, 0x80, 0x80, 0xE0, 0x80, 0x80, 0xE0, 0x80, 0x80, + 0xE0, 0x80, 0x80, 0xE0, 0x80, 0x80, 0xE0, 0x80, 0x80, 0xE0, 0x80, 0x80, + 0xE0, 0x80, 0x80, 0xE0, 0x80, 0x80, 0xE0, 0x80, 0x80, 0xE0, 0x80, 0x80, + 0xE0, 0x80, 0x80, 0xE0, +}; +#define UTF8_3BYTE_SPAN 0x0000FFFFFFFFFFFFull + +TERM_TARGET_AVX512 +static inline __m512i utf8_3byte_decode(__m512i packed) +{ + __m512i b2 = _mm512_and_si512(packed, _mm512_set1_epi32(0x003F)); + __m512i b1 = _mm512_and_si512(_mm512_srli_epi32(packed, 2), _mm512_set1_epi32(0x0FC0)); + __m512i b0 = _mm512_and_si512(_mm512_srli_epi32(packed, 4), _mm512_set1_epi32(0xF000)); + return _mm512_ternarylogic_epi32(b2, b1, b0, 0xFE); +} + +struct width_ranges { + __m512i lo[TERM_WIDTH_SLOTS]; + __m512i span[TERM_WIDTH_SLOTS]; +}; + +TERM_TARGET_AVX512 +static inline struct width_ranges width_ranges_load(const struct width_cache *wc) +{ + struct width_ranges r; + for (int j = 0; j < TERM_WIDTH_SLOTS; j++) { + r.lo[j] = _mm512_set1_epi32((int)wc->lo[j]); + r.span[j] = _mm512_set1_epi32((int)wc->span[j]); + } + return r; +} + +TERM_TARGET_AVX512 +static inline __mmask16 width_ranges_cover(const struct width_ranges *r, __m512i cps) +{ + return _mm512_cmple_epu32_mask(_mm512_sub_epi32(cps, r->lo[0]), r->span[0]) + | _mm512_cmple_epu32_mask(_mm512_sub_epi32(cps, r->lo[1]), r->span[1]) + | _mm512_cmple_epu32_mask(_mm512_sub_epi32(cps, r->lo[2]), r->span[2]) + | _mm512_cmple_epu32_mask(_mm512_sub_epi32(cps, r->lo[3]), r->span[3]); +} + +TERM_TARGET_AVX512 +static inline void store_wide_cells(uint64_t * restrict dst, __m512i base, __m512i cps) +{ +#define TERM_WIDE_PAIR(k) _mm512_storeu_si512((void *)(dst + 8 * (k)), \ + _mm512_mask_permutexvar_epi32(base, 0x0101, \ + _mm512_setr_epi32((k) * 2, 0, 0, 0, 0, 0, 0, 0, \ + (k) * 2 + 1, 0, 0, 0, 0, 0, 0, 0), cps)) + TERM_WIDE_PAIR(0); TERM_WIDE_PAIR(1); TERM_WIDE_PAIR(2); TERM_WIDE_PAIR(3); + TERM_WIDE_PAIR(4); TERM_WIDE_PAIR(5); TERM_WIDE_PAIR(6); TERM_WIDE_PAIR(7); +#undef TERM_WIDE_PAIR +} + +TERM_TARGET_AVX512 +static inline void store_narrow_cells(uint64_t * restrict dst, __m512i base, __m512i cps) +{ +#define TERM_NARROW_QUAD(k) _mm512_storeu_si512((void *)(dst + 8 * (k)), \ + _mm512_mask_permutexvar_epi32(base, 0x1111, \ + _mm512_setr_epi32((k) * 4, 0, 0, 0, (k) * 4 + 1, 0, 0, 0, \ + (k) * 4 + 2, 0, 0, 0, (k) * 4 + 3, 0, 0, 0), cps)) + TERM_NARROW_QUAD(0); TERM_NARROW_QUAD(1); TERM_NARROW_QUAD(2); TERM_NARROW_QUAD(3); +#undef TERM_NARROW_QUAD +} + +TERM_TARGET_AVX512 +static TERM_NOINLINE int +decode_3byte_blocks(uint64_t * restrict dst, const char * restrict buf, int len, int cols_left, + __m512i base_wide_pair, __m512i base_narrow_quad, + struct width_cache * restrict wc, + int * restrict out_cols, uint32_t * restrict last_cp) +{ + struct width_ranges ranges = width_ranges_load(wc); + __m512i shuffle = _mm512_loadu_si512(utf8_3byte_shuffle); + __m512i want = _mm512_loadu_si512(utf8_3byte_want); + __m512i bits = _mm512_loadu_si512(utf8_3byte_bits); + uint32_t last = *last_cp; + int i = 0, cx = 0; + + while (i + 48 <= len && cx + 32 <= cols_left && + ((unsigned char)buf[i + 45] & UTF8_LEAD3_MASK) == UTF8_LEAD3_TAG) { + __m512i in = _mm512_maskz_loadu_epi8(UTF8_3BYTE_SPAN, buf + i); + if (TERM_EXPECT(_mm512_test_epi8_mask(_mm512_xor_si512(in, want), bits) & + UTF8_3BYTE_SPAN, 0)) + break; + + __m512i cps = utf8_3byte_decode(_mm512_permutexvar_epi8(shuffle, in)); + if (_mm512_cmplt_epu32_mask(cps, _mm512_set1_epi32((int)UTF8_3BYTE_MIN)) || + _mm512_cmple_epu32_mask( + _mm512_sub_epi32(cps, _mm512_set1_epi32((int)UTF8_SURROGATE_LO)), + _mm512_set1_epi32((int)(UTF8_SURROGATE_HI - UTF8_SURROGATE_LO)))) + break; + __mmask16 covered = width_ranges_cover(&ranges, cps); + if (TERM_EXPECT(covered != 0xFFFF || wc->klass == 0, 0)) { + uint32_t cp[16]; + _mm512_storeu_si512(cp, cps); + if (!width_cache_extend(wc, cp, covered)) + break; + ranges = width_ranges_load(wc); + } + + if (wc->klass == 2) { + store_wide_cells(dst, base_wide_pair, cps); + dst += 64; + cx += 32; + } else { + store_narrow_cells(dst, base_narrow_quad, cps); + dst += 32; + cx += 16; + } + last = (uint32_t)_mm_extract_epi32(_mm512_extracti32x4_epi32(cps, 3), 3); + i += 48; + } + + *out_cols = cx; + *last_cp = last; + return i; +} + +/* decode_3byte_blocks() that writes the history format: a narrow block + * copies its 48 bytes, a wide block puts 0xF5 before each character. */ +TERM_TARGET_AVX512 +static TERM_NOINLINE int +text_3byte_blocks(uint8_t * restrict dst, const char * restrict buf, int len, int cols_left, + struct width_cache * restrict wc, int * restrict out_cols, + int * restrict out_bytes, bool * restrict wide, uint32_t * restrict last_cp) +{ + static const uint8_t spread[64] = { + 0, 0, 1, 2, 0, 3, 4, 5, 0, 6, 7, 8, 0, 9, 10, 11, + 0, 12, 13, 14, 0, 15, 16, 17, 0, 18, 19, 20, 0, 21, 22, 23, + 0, 24, 25, 26, 0, 27, 28, 29, 0, 30, 31, 32, 0, 33, 34, 35, + 0, 36, 37, 38, 0, 39, 40, 41, 0, 42, 43, 44, 0, 45, 46, 47, + }; + struct width_ranges ranges = width_ranges_load(wc); + __m512i shuffle = _mm512_loadu_si512(utf8_3byte_shuffle); + __m512i want = _mm512_loadu_si512(utf8_3byte_want); + __m512i bits = _mm512_loadu_si512(utf8_3byte_bits); + __m512i idx = _mm512_loadu_si512(spread); + __m512i prefix = _mm512_set1_epi8((char)0xF5); + uint32_t last = *last_cp; + int i = 0, cx = 0, o = 0; + + while (i + 48 <= len && cx + 32 <= cols_left && + ((unsigned char)buf[i + 45] & UTF8_LEAD3_MASK) == UTF8_LEAD3_TAG) { + __m512i in = _mm512_maskz_loadu_epi8(UTF8_3BYTE_SPAN, buf + i); + if (TERM_EXPECT(_mm512_test_epi8_mask(_mm512_xor_si512(in, want), bits) & + UTF8_3BYTE_SPAN, 0)) + break; + __m512i cps = utf8_3byte_decode(_mm512_permutexvar_epi8(shuffle, in)); + if (_mm512_cmplt_epu32_mask(cps, _mm512_set1_epi32((int)UTF8_3BYTE_MIN)) || + _mm512_cmple_epu32_mask( + _mm512_sub_epi32(cps, _mm512_set1_epi32((int)UTF8_SURROGATE_LO)), + _mm512_set1_epi32((int)(UTF8_SURROGATE_HI - UTF8_SURROGATE_LO)))) + break; + __mmask16 covered = width_ranges_cover(&ranges, cps); + if (TERM_EXPECT(covered != 0xFFFF || wc->klass == 0, 0)) { + uint32_t cp[16]; + _mm512_storeu_si512(cp, cps); + if (!width_cache_extend(wc, cp, covered)) + break; + ranges = width_ranges_load(wc); + } + if (wc->klass == 2) { + __m512i out = _mm512_mask_mov_epi8(_mm512_permutexvar_epi8(idx, in), + 0x1111111111111111ull, prefix); + _mm512_storeu_si512((void *)(dst + o), out); + o += 64; + cx += 32; + *wide = true; + } else { + _mm512_mask_storeu_epi8(dst + o, UTF8_3BYTE_SPAN, in); + o += 48; + cx += 16; + } + last = (uint32_t)_mm_extract_epi32(_mm512_extracti32x4_epi32(cps, 3), 3); + i += 48; + } + *out_cols = cx; + *out_bytes = o; + *last_cp = last; + return i; +} +#endif + +struct text_run_bytes term_text_run_bytes(uint8_t *restrict dst, const char *restrict buf, int len, + int max_cols, uint32_t last, struct width_cache *restrict wc) +{ + if (TERM_EXPECT(last == 0x200D || grapheme_is_handle(last), 0)) + return (struct text_run_bytes){ .last = last, .narrow = true }; + /* Unlike term_text_run(), this fills up to the edge and up to the end of + * buf: the cells path writes those characters one by one with the same + * result, and a text row should not settle for them. */ + int i = 0, cx = 0, o = 0, chars = 0; + bool wide = false, narrow = true; + (void)wc; +#if defined(TERM_KERNEL_AVX512) + bool blocks = term_use_avx512(); +#endif + while (i < len && cx < max_cols) { + unsigned char c = (unsigned char)buf[i]; + if (c >= 0x20 && c < 0x7f) { + int room = max_cols - cx < len - i ? max_cols - cx : len - i; + int n = 1; + while (n < room && n < 16 && (unsigned char)(buf[i + n] - 0x20) < 0x5f) + n++; + if (n == 16 && n < room) + n += (int)scan_printable_span(buf + i + 16, (size_t)(room - 16), 0x7f); + bytes_copy_small(dst + o, buf + i, n); + i += n; + o += n; + cx += n; + chars += n; + last = (unsigned char)buf[i - 1]; + continue; + } + if (c < 0x20) + break; +#if defined(TERM_KERNEL_AVX512) + if ((c & UTF8_LEAD3_MASK) == UTF8_LEAD3_TAG && len - i >= 48 && max_cols - cx >= 32 && + ((unsigned char)buf[i + 21] & UTF8_LEAD3_MASK) == UTF8_LEAD3_TAG && blocks) { + int cols, bytes; + bool block_wide = false; + int used = text_3byte_blocks(dst + o, buf + i, len - i, max_cols - cx, wc, + &cols, &bytes, &block_wide, &last); + if (used) { + i += used; + cx += cols; + o += bytes; + chars += used / 3; + wide |= block_wide; + narrow = false; + continue; + } + } +#endif + uint32_t cp; + int bytes = utf8_decode_one(buf + i, (size_t)(len - i), &cp); + if (!bytes) + break; + int w = bytes < 4 ? unicode_fast_class[cp] : term_text_class(cp); + if (w <= 0 || (bytes == 4 && can_extend_cluster(last, cp, w)) || cx + w > max_cols) + break; + if (w == 2) { + dst[o++] = 0xF5; + wide = true; + } + bytes_copy_small(dst + o, buf + i, bytes); + o += bytes; + cx += w; + chars++; + narrow = false; + last = cp; + i += bytes; + } + return (struct text_run_bytes){ i, cx, o, chars, last, wide, narrow }; +} + +#if defined(TERM_KERNEL_AVX512) +TERM_TARGET_AVX512 +static inline __m512i wide_pair_base(uint32_t fg, uint32_t bg, uint32_t flags) +{ + return _mm512_setr_epi32( + 0, fg, bg, flags | CELL_WIDE, + 0, fg, bg, (flags & ~CELL_WIDE) | CELL_TRAIL, + 0, fg, bg, flags | CELL_WIDE, + 0, fg, bg, (flags & ~CELL_WIDE) | CELL_TRAIL); +} + +TERM_TARGET_AVX512 +static inline int try_3byte_blocks(uint64_t * restrict dst, const char * restrict buf, + int len, int cols_left, uint32_t fg, uint32_t bg, + uint32_t flags, struct width_cache * restrict wc, + int * restrict out_cols, uint32_t * restrict last_cp) +{ + *out_cols = 0; + if (len < 48 || cols_left < 32) return 0; + if (((unsigned char)buf[0] & UTF8_LEAD3_MASK) != UTF8_LEAD3_TAG) return 0; + if (((unsigned char)buf[21] & UTF8_LEAD3_MASK) != UTF8_LEAD3_TAG) return 0; + if (((unsigned char)buf[45] & UTF8_LEAD3_MASK) != UTF8_LEAD3_TAG) return 0; + __m512i base_narrow_quad = _mm512_broadcast_i32x4(_mm_set_epi32((int)flags, (int)bg, (int)fg, 0)); + return decode_3byte_blocks(dst, buf, len, cols_left, wide_pair_base(fg, bg, flags), + base_narrow_quad, wc, out_cols, last_cp); +} + +/* Two wide cells side by side: cp and next_cp. */ +TERM_TARGET_AVX512 +static inline void store_wide_pair(uint64_t *dst, uint32_t fg, uint32_t bg, uint32_t flags, + uint32_t cp, uint32_t next_cp) +{ + __m512i cells = _mm512_mask_set1_epi32(wide_pair_base(fg, bg, flags), 0x0001, (int)cp); + cells = _mm512_mask_set1_epi32(cells, 0x0100, (int)next_cp); + _mm512_storeu_si512((void *)dst, cells); +} +#endif + +struct cell_bases { +#if defined(__x86_64__) || defined(_M_X64) + __m128i cell, wide, trail; +#elif defined(__aarch64__) + uint32x4_t cell, wide, trail; +#else + uint64_t fg_shifted, high, high_wide, high_trail; +#endif +}; + +static inline struct cell_bases cell_bases_make(uint32_t fg, uint32_t bg, uint32_t flags) +{ + struct cell_bases b; +#if defined(__x86_64__) || defined(_M_X64) + b.cell = _mm_set_epi32(flags, bg, fg, 0); + b.wide = _mm_set_epi32(flags | CELL_WIDE, bg, fg, 0); + b.trail = _mm_set_epi32((flags & ~CELL_WIDE) | CELL_TRAIL, bg, fg, 0); +#elif defined(__aarch64__) + b.cell = (uint32x4_t){ 0, fg, bg, (uint32_t)flags }; + b.wide = (uint32x4_t){ 0, fg, bg, (uint32_t)(flags | CELL_WIDE) }; + b.trail = (uint32x4_t){ 0, fg, bg, + (uint32_t)((flags & ~CELL_WIDE) | CELL_TRAIL) }; +#else + b.fg_shifted = (uint64_t)fg << 32; + b.high = ((uint64_t)flags << 32) | bg; + b.high_wide = ((uint64_t)(flags | CELL_WIDE) << 32) | bg; + b.high_trail = ((uint64_t)((flags & ~CELL_WIDE) | CELL_TRAIL) << 32) | bg; +#endif + return b; +} + +static inline uint64_t *store_cell(uint64_t * restrict dst, + const struct cell_bases *b, uint32_t cp) +{ +#if defined(__x86_64__) || defined(_M_X64) + _mm_storeu_si128((__m128i *)dst, _mm_or_si128(b->cell, _mm_cvtsi32_si128(cp))); +#elif defined(__aarch64__) + vst1q_u32((uint32_t *)(void *)dst, vsetq_lane_u32(cp, b->cell, 0)); +#else + dst[0] = b->fg_shifted | cp; + dst[1] = b->high; +#endif + return dst + 2; +} + +static inline uint64_t *store_cell_wide(uint64_t * restrict dst, + const struct cell_bases *b, uint32_t cp) +{ +#if defined(__x86_64__) || defined(_M_X64) + _mm_storeu_si128((__m128i *)dst, _mm_or_si128(b->wide, _mm_cvtsi32_si128(cp))); + _mm_storeu_si128((__m128i *)(dst + 2), b->trail); +#elif defined(__aarch64__) + vst1q_u32((uint32_t *)(void *)dst, vsetq_lane_u32(cp, b->wide, 0)); + vst1q_u32((uint32_t *)(void *)(dst + 2), b->trail); +#else + dst[0] = b->fg_shifted | cp; + dst[1] = b->high_wide; + dst[2] = b->fg_shifted; + dst[3] = b->high_trail; +#endif + return dst + 4; +} + +#if defined(TERM_KERNEL_AVX2) +/* + * Decode runs of 3-byte UTF-8 four code points at a time. A quad stops the + * run when a byte is not a 3-byte sequence, a code point is overlong or a + * surrogate, or a code point has no fast width class. The scalar loop then + * handles that code point, so both paths store the same cells. + */ +TERM_TARGET_AVX2 +static inline int decode_3byte_quads(uint64_t **dst_io, const char *buf, int len, + int cols_left, const struct cell_bases *b, + int *out_cols, uint32_t *last) +{ + const __m128i want = _mm_setr_epi8((char)0xE0, (char)0x80, (char)0x80, + (char)0xE0, (char)0x80, (char)0x80, + (char)0xE0, (char)0x80, (char)0x80, + (char)0xE0, (char)0x80, (char)0x80, 0, 0, 0, 0); + const __m128i bits = _mm_setr_epi8((char)0xF0, (char)0xC0, (char)0xC0, + (char)0xF0, (char)0xC0, (char)0xC0, + (char)0xF0, (char)0xC0, (char)0xC0, + (char)0xF0, (char)0xC0, (char)0xC0, 0, 0, 0, 0); + const __m128i shuffle = _mm_setr_epi8(2, 1, 0, -1, 5, 4, 3, -1, + 8, 7, 6, -1, 11, 10, 9, -1); + uint64_t *dst = *dst_io; + int i = 0, cx = 0; + while (i + 16 <= len && cx + 8 <= cols_left) { + __m128i in = _mm_loadu_si128((const __m128i *)(buf + i)); + __m128i ok = _mm_cmpeq_epi8(_mm_and_si128(_mm_xor_si128(in, want), bits), + _mm_setzero_si128()); + if (_mm_movemask_epi8(ok) != 0xFFFF) + break; + __m128i packed = _mm_shuffle_epi8(in, shuffle); + __m128i cps = _mm_or_si128( + _mm_or_si128(_mm_and_si128(packed, _mm_set1_epi32(0x003F)), + _mm_and_si128(_mm_srli_epi32(packed, 2), _mm_set1_epi32(0x0FC0))), + _mm_and_si128(_mm_srli_epi32(packed, 4), _mm_set1_epi32(0xF000))); + __m128i bad = _mm_or_si128( + _mm_cmplt_epi32(cps, _mm_set1_epi32((int)UTF8_3BYTE_MIN)), + _mm_cmpeq_epi32(_mm_and_si128(cps, _mm_set1_epi32(0xF800)), + _mm_set1_epi32((int)UTF8_SURROGATE_LO))); + if (_mm_movemask_epi8(bad)) + break; + alignas(16) uint32_t cp[4]; + _mm_store_si128((__m128i *)cp, cps); + uint8_t k0 = unicode_fast_class[cp[0]], k1 = unicode_fast_class[cp[1]]; + uint8_t k2 = unicode_fast_class[cp[2]], k3 = unicode_fast_class[cp[3]]; + if (!k0 || !k1 || !k2 || !k3) + break; + const uint8_t k[4] = { k0, k1, k2, k3 }; + for (int j = 0; j < 4; j++) { + if (k[j] == 1) { + dst = store_cell(dst, b, cp[j]); + cx++; + } else { + dst = store_cell_wide(dst, b, cp[j]); + cx += 2; + } + } + *last = cp[3]; + i += 12; + } + *dst_io = dst; + *out_cols = cx; + return i; +} +#endif + +/* One body for every level. isa is a constant in each copy below, so each + * copy keeps only its own kernels. */ +static inline TERM_ALWAYS_INLINE struct text_run_result +text_run_body(struct cell *restrict row, const char *restrict buf, int len, int max_cols, + uint32_t fg, uint32_t bg, uint32_t flags, uint32_t last, + struct width_cache *restrict wc, int isa) +{ + int i = 0; + int cx = 0; + if (TERM_EXPECT(last == 0x200D || grapheme_is_handle(last), 0)) + return (struct text_run_result){0, 0, last}; + + const struct cell_bases bases = cell_bases_make(fg, bg, flags); + uint64_t *dst = (uint64_t *)row; + (void)wc; + (void)isa; + +#if defined(TERM_KERNEL_AVX512) + if (isa == TERM_ISA_AVX512) { + int cols; + i += try_3byte_blocks(dst, buf, len, max_cols, fg, bg, flags, wc, &cols, &last); + cx += cols; + dst += 2 * cols; + } +#endif + + while (i + 4 <= len && cx + 2 <= max_cols) { + unsigned char c = (unsigned char)buf[i]; + if (c >= 0x20 && c < 0x7f) { + int ascii_at = cx; + dst = store_cell(dst, &bases, c); + last = c; + cx++; + i++; + while (i + 4 <= len && cx + 2 <= max_cols) { + unsigned char next_c = (unsigned char)buf[i]; + if (next_c < 0x20 || next_c >= 0x7f) break; + dst = store_cell(dst, &bases, next_c); + last = next_c; + cx++; + i++; + } +#if defined(TERM_KERNEL_AVX512) + if (isa == TERM_ISA_AVX512 && ascii_at == 0) { + int cols; + i += try_3byte_blocks(dst, buf + i, len - i, max_cols - cx, + fg, bg, flags, wc, &cols, &last); + cx += cols; + dst += 2 * cols; + } +#endif + (void)ascii_at; + continue; + } + if (c < 0x20) { + break; + } + +#if defined(TERM_KERNEL_AVX2) + if (isa == TERM_ISA_AVX2 && + TERM_EXPECT((c & UTF8_LEAD3_MASK) == UTF8_LEAD3_TAG && i + 16 <= len && + ((unsigned char)buf[i + 9] & UTF8_LEAD3_MASK) == UTF8_LEAD3_TAG, 0)) { + int cols; + int used = decode_3byte_quads(&dst, buf + i, len - i, max_cols - cx, + &bases, &cols, &last); + i += used; + cx += cols; + if (used) + continue; + } +#endif + uint32_t cp; + int bytes = utf8_decode_one(buf + i, (size_t)(len - i), &cp); + if (!bytes) break; + int w = bytes < 4 ? unicode_fast_class[cp] : term_text_class(cp); + if (w <= 0 || (bytes == 4 && can_extend_cluster(last, cp, w))) break; +#if defined(TERM_KERNEL_AVX512) + if (isa == TERM_ISA_AVX512 && bytes == 3 && w == 2 && i + 6 <= len && cx + 4 <= max_cols) { + uint32_t next_cp; + if (utf8_decode_one(buf + i + 3, (size_t)(len - i - 3), &next_cp) == 3 && + unicode_fast_class[next_cp] == 2) { + store_wide_pair(dst, fg, bg, flags, cp, next_cp); + dst += 8; + last = next_cp; + cx += 4; + i += 6; + continue; + } + } +#endif + if (w == 1) { + dst = store_cell(dst, &bases, cp); + cx++; + } else { + dst = store_cell_wide(dst, &bases, cp); + cx += 2; + } + last = cp; + i += bytes; + } + + return (struct text_run_result){i, cx, last}; +} + +#define TEXT_RUN_PARAMS struct cell *restrict row, const char *restrict buf, int len, \ + int max_cols, uint32_t fg, uint32_t bg, uint32_t flags, uint32_t last, \ + struct width_cache *restrict wc +#define TEXT_RUN_ARGS row, buf, len, max_cols, fg, bg, flags, last, wc + +#if defined(TERM_KERNEL_AVX512) +TERM_TARGET_AVX512 +static TERM_NOINLINE struct text_run_result text_run_avx512(TEXT_RUN_PARAMS) +{ + return text_run_body(TEXT_RUN_ARGS, TERM_ISA_AVX512); +} +#endif + +#if defined(TERM_KERNEL_AVX2) +TERM_TARGET_AVX2 +static TERM_NOINLINE struct text_run_result text_run_avx2(TEXT_RUN_PARAMS) +{ + return text_run_body(TEXT_RUN_ARGS, TERM_ISA_AVX2); +} +#endif + +static TERM_NOINLINE struct text_run_result text_run_base(TEXT_RUN_PARAMS) +{ + return text_run_body(TEXT_RUN_ARGS, TERM_ISA_BASE); +} + +struct text_run_result term_text_run(TEXT_RUN_PARAMS) +{ +#if defined(TERM_KERNEL_AVX512) + if (term_use_avx512()) + return text_run_avx512(TEXT_RUN_ARGS); +#endif +#if defined(TERM_KERNEL_AVX2) + if (term_use_avx2()) + return text_run_avx2(TEXT_RUN_ARGS); +#endif + return text_run_base(TEXT_RUN_ARGS); +} diff --git a/src/text_run.h b/src/text_run.h new file mode 100644 index 0000000..d546062 --- /dev/null +++ b/src/text_run.h @@ -0,0 +1,90 @@ +#ifndef TERM_TEXT_RUN_H +#define TERM_TEXT_RUN_H + +#include "cell.h" +#include "unicode/unicode.h" +#include "unicode/grapheme.h" + +#define UTF8_LEAD_MIN 0xc2 +#define UTF8_LEAD_MAX 0xf4 +#define UTF8_LEAD3_TAG 0xE0 +#define UTF8_LEAD3_MASK 0xF0 + +#define TERM_WIDTH_SLOTS 4 +struct width_cache { + uint32_t lo[TERM_WIDTH_SLOTS]; + uint32_t span[TERM_WIDTH_SLOTS]; + uint8_t klass; + uint8_t next; +}; + +/* term_new() builds the shared tables before any terminal can use them. */ +bool term_tables_ready(void); +void term_text_tables_init(void); +void term_snapshot_tables_init(void); + +int term_text_class(uint32_t cp); + +struct text_run_result { + int bytes; + int cols; + uint32_t last; +}; + +struct text_run_result term_text_run(struct cell *restrict row, const char *restrict buf, + int len, int max_cols, uint32_t fg, uint32_t bg, + uint32_t flags, uint32_t last, struct width_cache *restrict wc); + +struct text_run_bytes { + int bytes; /* input used */ + int cols; + int out; /* bytes written */ + int chars; + uint32_t last; + bool wide, narrow; /* narrow: every character was one ASCII byte */ +}; + +/* term_text_run() with the same rules, that writes the text in the history + * format instead of cells: raw UTF-8, a wide character after 0xF5. dst has + * room for 4 bytes per column. */ +struct text_run_bytes term_text_run_bytes(uint8_t *restrict dst, const char *restrict buf, int len, + int max_cols, uint32_t last, struct width_cache *restrict wc); + +static inline bool joins_cluster(uint32_t cp, int klass) +{ + if (cp < 0x0300) + return false; + if (klass == 0 || grapheme_emoji_modifier(cp)) + return true; + + return klass == 1 && unicode_spacing_mark(cp); +} + +static inline bool can_extend_cluster(uint32_t last_char, uint32_t cp, int klass) +{ + if (klass == 0) return true; + if (last_char == 0x200D || grapheme_is_handle(last_char)) return true; + if (cp - 0x1F1E6u <= 0x19u && grapheme_regional_indicator(cp)) + return true; + return joins_cluster(cp, klass); +} + +static inline bool cluster_contextual(uint32_t cp) +{ + return unicode_spacing_mark(cp) || grapheme_emoji_modifier(cp) || + (cp - 0x1F1E6u <= 0x19u && grapheme_regional_indicator(cp)); +} + +static inline bool cluster_joins(const uint32_t *cps, int len, + uint32_t cp, int klass) +{ + if (joins_cluster(cp, klass)) + return true; + if (cps[len - 1] == 0x200D) + return unicode_extended_pictographic(cp); + return len == 1 && cp - 0x1F1E6u <= 0x19u && + grapheme_regional_indicator(cp) && + grapheme_regional_indicator(cps[0]); +} + +#endif diff --git a/tests/cross_test.sh b/tests/cross_test.sh new file mode 100644 index 0000000..6c0efa2 --- /dev/null +++ b/tests/cross_test.sh @@ -0,0 +1,24 @@ +#!/usr/bin/env bash +# The term core compiles as strict C11 for the platforms a host may embed +# it on. zig cc supplies the cross compilers. The aarch64 targets build the +# NEON paths, and windows-gnu builds the Windows path of compat.h. The +# amalgamation, one term.c, compiles the same way. +set -euo pipefail +root=${1:-$(dirname "$0")/..} +cd "$root/src" +command -v zig >/dev/null || { echo "zig not found; skipping"; exit 77; } + +out=$(mktemp -d) +trap 'rm -rf "$out"' EXIT +python3 "$root/tools/amalgamate.py" "$root" "$out/amalgamation" +status=0 +for target in aarch64-linux-gnu x86_64-macos aarch64-macos x86_64-windows-gnu; do + for src in *.c osc/*.c; do + zig cc -target "$target" -std=c11 -O2 -Wall -Wextra -Wpedantic -Werror \ + -I. -I../include/term -c "$src" -o "$out/obj.o" || { echo "$target: $src fails" >&2; status=1; } + done + zig cc -target "$target" -std=c11 -O2 -Wall -Wextra -Wpedantic -Werror \ + -c "$out/amalgamation/term.c" -o "$out/obj.o" || + { echo "$target: the amalgamation fails" >&2; status=1; } +done +exit $status diff --git a/tests/dcs_test.c b/tests/dcs_test.c new file mode 100644 index 0000000..0e67f78 --- /dev/null +++ b/tests/dcs_test.c @@ -0,0 +1,475 @@ +#include "term_api.h" +#include <assert.h> +#include <stdbool.h> +#include <stdio.h> +#include <string.h> + +static char response[16384]; +static size_t response_len; +static void respond(void *ud, const char *data, size_t len) +{ + (void)ud; + if (len >= sizeof(response)) len = sizeof(response) - 1; + memcpy(response, data, len); + response[len] = 0; + response_len = len; +} +static void write_seq(struct term *t, const char *s) +{ + for (; *s; s++) tt_write(t, s, 1); +} +static const char *row_text(struct term *t, int r, char *buf) +{ + int n = 0; + for (int c = 0; c < tt_cols(t); c++) + buf[n++] = (char)tt_view_cell(t, c, r)->text; + while (n > 0 && buf[n - 1] == ' ') n--; + buf[n] = 0; + return buf; +} +static bool row_is(struct term *t, int r, const char *want) +{ + char got[TERM_MAX_COLS + 1]; + if (strcmp(row_text(t, r, got), want)) { + fprintf(stderr, "row %d: got \"%s\" want \"%s\"\n", r, got, want); + return false; + } + return true; +} +static struct term *new_term(int cols, int rows) +{ + struct term *t = tt_new_opts(cols, rows, &(struct term_callbacks){ .reply = respond }); + assert(t); + response_len = 0; + response[0] = 0; + return t; +} +static void expect_response(const char *want) +{ + if (response_len != strlen(want) || memcmp(response, want, response_len)) { + fprintf(stderr, "response: got len %zu \"%s\" want \"%s\"\n", + response_len, response, want); + assert(!"response mismatch"); + } +} +static void expect_no_response(void) +{ + assert(response_len == 0); +} + +static void hex_encode(char *out, const char *s) +{ + for (; *s; s++) { + snprintf(out, 3, "%02x", (unsigned char)*s); + out += 2; + } + *out = 0; +} + +static void assert_rqss(struct term *t, const char *spec, const char *payload) +{ + char req[128], want[256]; + snprintf(req, sizeof(req), "\033P$q%s\033\\", spec); + snprintf(want, sizeof(want), "\033P1$r%s\033\\", payload); + response_len = 0; + write_seq(t, req); + expect_response(want); +} +static void assert_rqss_invalid(struct term *t, const char *spec) +{ + char req[128]; + snprintf(req, sizeof(req), "\033P$q%s\033\\", spec); + response_len = 0; + write_seq(t, req); + expect_response("\033P0$r\033\\"); +} + +static void test_decrqss_sgr(void) +{ + struct term *t = new_term(80, 24); + assert_rqss(t, "m", "m"); + write_seq(t, "\033[1;31m"); + assert_rqss(t, "m", "0;1;31m"); + term_free(t); + + t = new_term(80, 24); + write_seq(t, "\033[4:3m"); + assert_rqss(t, "m", "0;4:3m"); + term_free(t); + + t = new_term(80, 24); + write_seq(t, "\033[38;2;10;20;30m"); + assert_rqss(t, "m", "0;38:2:10:20:30m"); + term_free(t); +} + +static void test_decrqss_margins_and_page(void) +{ + struct term *t = new_term(80, 24); + assert_rqss(t, "r", "1;24r"); + assert_rqss(t, "t", "24t"); + assert_rqss(t, "$|", "80$|"); + assert_rqss(t, "*|", "24*|"); + write_seq(t, "\033[3;10r"); + assert_rqss(t, "r", "3;10r"); + term_free(t); + + t = new_term(80, 24); + assert_rqss(t, "s", "1;80s"); + write_seq(t, "\033[?69h\033[5;20s"); + assert_rqss(t, "s", "5;20s"); + term_free(t); +} + +static void test_decrqss_cursor_style(void) +{ + struct term *t = new_term(80, 24); + assert_rqss(t, " q", "1 q"); + write_seq(t, "\033[5 q"); + assert_rqss(t, " q", "5 q"); + write_seq(t, "\033[4 q"); + assert_rqss(t, " q", "4 q"); + term_free(t); +} + +static void test_decrqss_misc(void) +{ + struct term *t = new_term(80, 24); + assert_rqss(t, "\"p", "64;1\"p"); + assert_rqss(t, "\"q", "0\"q"); + assert_rqss(t, "$}", "0$}"); + assert_rqss(t, "$~", "0$~"); + assert_rqss(t, "*x", "0*x"); + write_seq(t, "\033[2*x"); + assert_rqss(t, "*x", "2*x"); + term_free(t); +} + +static void test_decrqss_invalid(void) +{ + struct term *t = new_term(80, 24); + static const char *bad[] = { + "){", ",|", ",}", ">0f", ">0m", ">0t", "z", "ABC", + }; + for (size_t i = 0; i < sizeof(bad) / sizeof(bad[0]); i++) + assert_rqss_invalid(t, bad[i]); + + response_len = 0; + write_seq(t, "\033P$x\033\\"); + expect_no_response(); + term_free(t); +} + +static void assert_tcap(struct term *t, const char *name, const char *value) +{ + char hn[128], hv[512], req[256], want[1024]; + hex_encode(hn, name); + hex_encode(hv, value); + snprintf(req, sizeof(req), "\033P+q%s\033\\", hn); + snprintf(want, sizeof(want), "\033P1+r%s=%s\033\\", hn, hv); + response_len = 0; + write_seq(t, req); + expect_response(want); +} + +static void test_xtgettcap(void) +{ + struct term *t = new_term(80, 24); + assert_tcap(t, "kind", "\033[1;2B"); + assert_tcap(t, "TN", "wsap_term"); + assert_tcap(t, "name", "wsap_term"); + assert_tcap(t, "Co", "16777216"); + assert_tcap(t, "colors", "16777216"); + assert_tcap(t, "RGB", "8/8/8"); + assert_tcap(t, "kbs", "\177"); + assert_tcap(t, "kcbt", "\033[Z"); + assert_tcap(t, "kcuu1", "\033OA"); + assert_tcap(t, "kcud1", "\033OB"); + assert_tcap(t, "kcuf1", "\033OC"); + assert_tcap(t, "kcub1", "\033OD"); + assert_tcap(t, "khome", "\033OH"); + assert_tcap(t, "kend", "\033OF"); + assert_tcap(t, "kent", "\033OM"); + assert_tcap(t, "kich1", "\033[2~"); + assert_tcap(t, "kdch1", "\033[3~"); + assert_tcap(t, "kpp", "\033[5~"); + assert_tcap(t, "knp", "\033[6~"); + assert_tcap(t, "kf1", "\033OP"); + assert_tcap(t, "kf2", "\033OQ"); + assert_tcap(t, "kf3", "\033OR"); + assert_tcap(t, "kf4", "\033OS"); + assert_tcap(t, "kf5", "\033[15~"); + assert_tcap(t, "kf6", "\033[17~"); + assert_tcap(t, "kf7", "\033[18~"); + assert_tcap(t, "kf8", "\033[19~"); + assert_tcap(t, "kf9", "\033[20~"); + assert_tcap(t, "kf10", "\033[21~"); + assert_tcap(t, "kf11", "\033[23~"); + assert_tcap(t, "kf12", "\033[24~"); + assert_tcap(t, "kf13", "\033[1;2P"); + assert_tcap(t, "kf24", "\033[24;2~"); + assert_tcap(t, "kUP", "\033[1;2A"); + assert_tcap(t, "kDN", "\033[1;2B"); + assert_tcap(t, "kLFT", "\033[1;2D"); + assert_tcap(t, "kRIT", "\033[1;2C"); + assert_tcap(t, "kmous", "\033[<"); + assert_tcap(t, "kri", "\033[1;2A"); + assert_tcap(t, "kxIN", "\033[I"); + assert_tcap(t, "kxOUT", "\033[O"); + term_free(t); +} + +static void test_xtgettcap_multi(void) +{ + struct term *t = new_term(80, 24); + char hk[16], htn[16], hb[16], hv[32], req[256], want[1024]; + hex_encode(hk, "kind"); + hex_encode(htn, "TN"); + hex_encode(hb, "BOGUS"); + hex_encode(hv, "\033[1;2B"); + + snprintf(req, sizeof(req), "\033P+q%s;%s\033\\", hk, htn); + snprintf(want, sizeof(want), "\033P1+r%s=%s;%s=%s\033\\", + hk, hv, htn, "777361705f7465726d"); + response_len = 0; + write_seq(t, req); + expect_response(want); + + snprintf(req, sizeof(req), "\033P+q%s;%s;%s\033\\", hk, hb, htn); + snprintf(want, sizeof(want), "\033P1+r%s=%s;%s\033\\", hk, hv, hb); + response_len = 0; + write_seq(t, req); + expect_response(want); + assert(!strstr(response, htn)); + + snprintf(req, sizeof(req), "\033P+q%s\033\\", hb); + snprintf(want, sizeof(want), "\033P0+r%s\033\\", hb); + response_len = 0; + write_seq(t, req); + expect_response(want); + term_free(t); +} + +static void test_xtgettcap_malformed(void) +{ + struct term *t = new_term(80, 24); + static const char *bad[] = {"6b696", "zz", "6g", "6b69 6e"}; + for (size_t i = 0; i < sizeof(bad) / sizeof(bad[0]); i++) { + char req[128]; + snprintf(req, sizeof(req), "\033P+q%s\033\\", bad[i]); + response_len = 0; + write_seq(t, req); + assert(response_len > 4); + assert(!memcmp(response, "\033P0+r", 5)); + assert(response[response_len - 2] == '\033'); + assert(response[response_len - 1] == '\\'); + } + term_free(t); +} + +static void test_xtsettcap(void) +{ + struct term *t = new_term(80, 24); + char hn[32]; + hex_encode(hn, "myterm"); + char req[128]; + snprintf(req, sizeof(req), "\033P+p%s\033\\", hn); + response_len = 0; + write_seq(t, req); + expect_no_response(); + assert_tcap(t, "TN", "myterm"); + term_free(t); +} + +static void test_xtgetxres(void) +{ + struct term *t = new_term(80, 24); + char hn[64], req[128], want[256]; + hex_encode(hn, "allowTcapOps"); + snprintf(req, sizeof(req), "\033P+Q%s\033\\", hn); + snprintf(want, sizeof(want), "\033P0+R%s\033\\", hn); + response_len = 0; + write_seq(t, req); + expect_response(want); + term_free(t); +} + +static void test_decrsps_cir(void) +{ + struct term *t = new_term(80, 24); + write_seq(t, "\033P1$t3;5;1;I;@;H;00;@;BBBB\033\\"); + assert(tt_cursor_y(t) == 2 && tt_cursor_x(t) == 4); + + write_seq(t, "Z"); + const struct term_cell *cell = tt_cell(t, 4, 2); + assert(cell->text == 'Z'); + assert(cell->attrs & TERM_ATTR_BOLD); + assert(cell->fg == TERM_COLOR_BG); + term_free(t); + + t = new_term(80, 24); + write_seq(t, "\033P1$t3;5;1;@;@;H;00;@;BBBB\033\\"); + write_seq(t, "\033[1;80Hab"); + assert(tt_cell(t, 79, 0)->text == 'a'); + assert(tt_cell(t, 0, 1)->text == 'b'); + term_free(t); + + t = new_term(80, 24); + write_seq(t, "\033[5;15r"); + write_seq(t, "\033P1$t3;5;1;@;@;I;00;@;BBBB\033\\"); + write_seq(t, "\033[1;1H"); + assert(tt_cursor_y(t) == 4 && tt_cursor_x(t) == 0); + term_free(t); +} + +static void test_decrsps_tabsr(void) +{ + struct term *t = new_term(80, 24); + write_seq(t, "\033P2$t4/10\033\\"); + write_seq(t, "\033[1;1H"); + write_seq(t, "\t"); + assert(tt_cursor_x(t) == 3); + write_seq(t, "\t"); + assert(tt_cursor_x(t) == 9); + write_seq(t, "\t"); + assert(tt_cursor_x(t) == 79); + term_free(t); +} + +static void test_synchronized_output(void) +{ + struct term *t = new_term(80, 24); + write_seq(t, "\033P=1s\033\\"); + assert(term_modes(t).synchronized); + write_seq(t, "\033P=2s\033\\"); + assert(!term_modes(t).synchronized); + + write_seq(t, "\033P=10s\033\\"); + assert(!term_modes(t).synchronized); + write_seq(t, "\033P=1;s\033\\"); + assert(term_modes(t).synchronized); + write_seq(t, "\033P=xs\033\\"); + assert(term_modes(t).synchronized); + write_seq(t, "\033P=2;s\033\\"); + assert(!term_modes(t).synchronized); + term_free(t); +} + +static void test_ignored_dcs(void) +{ + static const char *seqs[] = { + "\033P0;1|17/0d\033\\", + "\033P0!u%5\033\\", + "\033P1!uA\033\\", + "\033P1;0;1;10;0;2;0;1{~\033\\", + "\033P1;0;0q#0~~\033\\", + "\033P1pS(E)\033\\", + "\033P?~junk\033\\", + }; + for (size_t i = 0; i < sizeof(seqs) / sizeof(seqs[0]); i++) { + struct term *t = new_term(80, 24); + write_seq(t, "A"); + write_seq(t, seqs[i]); + write_seq(t, "B"); + assert(row_is(t, 0, "AB")); + assert(tt_cursor_x(t) == 2 && tt_cursor_y(t) == 0); + expect_no_response(); + term_free(t); + } +} + +static void test_xtversion(void) +{ + struct term *t = new_term(80, 24); + response_len = 0; + write_seq(t, "\033[>0q"); + expect_response("\033P>|wsap_term\033\\"); + term_free(t); +} + +static void test_bel_terminator(void) +{ + struct term *t = new_term(80, 24); + response_len = 0; + write_seq(t, "\033P+q6b696e64\007"); + expect_response("\033P1+r6b696e64=1b5b313b3242\033\\"); + + response_len = 0; + write_seq(t, "\033P$qm\007"); + expect_response("\033P1$rm\033\\"); + term_free(t); +} + +/* A query far past the old 256-byte DCS buffer is answered in full, + * whether it arrives byte by byte or in one write. */ +static void test_dcs_long_query(void) +{ + enum { NAMES = 60 }; + char hk[16], hv[32]; + hex_encode(hk, "kind"); + hex_encode(hv, "\033[1;2B"); + static char req[4096], want[8192]; + int rn = snprintf(req, sizeof(req), "\033P+q"); + int wn = snprintf(want, sizeof(want), "\033P1+r"); + for (int i = 0; i < NAMES; i++) { + rn += snprintf(req + rn, sizeof(req) - (size_t)rn, "%s%s", i ? ";" : "", hk); + wn += snprintf(want + wn, sizeof(want) - (size_t)wn, "%s%s=%s", + i ? ";" : "", hk, hv); + } + snprintf(req + rn, sizeof(req) - (size_t)rn, "\033\\"); + snprintf(want + wn, sizeof(want) - (size_t)wn, "\033\\"); + assert(strlen(req) > 256); + + struct term *t = new_term(80, 24); + write_seq(t, req); + expect_response(want); + response_len = 0; + tt_write(t, req, strlen(req)); + expect_response(want); + write_seq(t, "X"); + assert(row_is(t, 0, "X")); + term_free(t); +} + +static void test_dcs_abort_c0(void) +{ + struct term *t = new_term(80, 24); + response_len = 0; + write_seq(t, "\033P+q6b69\nX"); + expect_no_response(); + assert(row_is(t, 1, "X")); + term_free(t); +} + +static void test_dcs_abort_esc(void) +{ + struct term *t = new_term(80, 24); + response_len = 0; + write_seq(t, "\033P$qm\033[5n"); + expect_response("\033[0n"); + term_free(t); +} + +int main(void) +{ + test_decrqss_sgr(); + test_decrqss_margins_and_page(); + test_decrqss_cursor_style(); + test_decrqss_misc(); + test_decrqss_invalid(); + test_xtgettcap(); + test_xtgettcap_multi(); + test_xtgettcap_malformed(); + test_xtsettcap(); + test_xtgetxres(); + test_decrsps_cir(); + test_decrsps_tabsr(); + test_synchronized_output(); + test_ignored_dcs(); + test_xtversion(); + test_bel_terminator(); + test_dcs_long_query(); + test_dcs_abort_c0(); + test_dcs_abort_esc(); + return 0; +} diff --git a/tests/extern/README.md b/tests/extern/README.md new file mode 100644 index 0000000..cff42ed --- /dev/null +++ b/tests/extern/README.md @@ -0,0 +1,43 @@ +# Imported test suites + +This directory holds test suites imported from external projects. These are +kept for reference and cross-checking against our own terminal emulator's +behavior. On Linux, `make test` builds and runs the adapted Kitty parser suite. + +## kitty/ + +Vendored from <https://github.com/kovidgoyal/kitty>, commit `f13c8cd4`. + +### Running the adapted suite + +```sh +meson test -C build --suite kitty +``` + +This builds `kitty_bridge` (a shared library compiled from +the project's terminal sources), then runs the +Kitty parser test suite against it via a Python compatibility layer. + +### Files + +- `kitty/parser.py` — Kitty's VT/CSI parser test suite (`kitty_tests/parser.py`), + with imports patched to use the local compatibility modules below. +- `kitty_parser.py` — Pure-Python VT parser that emits Kitty-style callback + events while feeding every byte to the C terminal via `screen.write()`. + Maintains state across calls for split-write (threading) tests. +- `screen.py` — `ctypes` wrapper around `libterm.so` providing the Kitty + `Screen`, `Cursor`, `Line`, `CellCursor`, and `Callbacks` API. +- `fast_data_types.py` — Pure-Python replacements for Kitty's C-extension + helpers (base64, UTF-8 decode, constants). +- `kitty_harness.py` — `BaseTest`, `CmdDump`, and `parse_bytes` re-export. +- `__init__.py` — Package marker. + +### Test status (14 tests) + +All 14 tests pass. Features implemented for Kitty parity include DEC +Special Graphics charset translation (G0/G1, SO/SI), literal `\t` cells, +LNM-default-off linefeed semantics, CSI insert-character and REP edge +cases, DECRQM mode reports, DECRQSS (DCS `$q`) and XTGETTCAP (DCS `+q`) +responses, DECCARA/DECSACE (CSI `$r` / `*x`), secondary device attributes +(CSI `=c`), XTWINOPS cell-size report (CSI `14t`), underline styles 0–5, +and SGR underline color (58/59) including colon-form subparameters. diff --git a/tests/extern/kitty/.gitignore b/tests/extern/kitty/.gitignore new file mode 100644 index 0000000..79a6f1c --- /dev/null +++ b/tests/extern/kitty/.gitignore @@ -0,0 +1,2 @@ +__pycache__/ +libterm.so diff --git a/tests/extern/kitty/__init__.py b/tests/extern/kitty/__init__.py new file mode 100644 index 0000000..5dddbaf --- /dev/null +++ b/tests/extern/kitty/__init__.py @@ -0,0 +1,9 @@ +"""Package marker for the Kitty parser test harness. + +Re-exports :class:`BaseTest`, :class:`CmdDump`, and ``parse_bytes`` from +:mod:`kitty_harness`. +""" + +from kitty_harness import BaseTest, CmdDump, parse_bytes + +__all__ = ['BaseTest', 'CmdDump', 'parse_bytes'] diff --git a/tests/extern/kitty/fast_data_types.py b/tests/extern/kitty/fast_data_types.py new file mode 100644 index 0000000..c8c519d --- /dev/null +++ b/tests/extern/kitty/fast_data_types.py @@ -0,0 +1,237 @@ +"""Stubs for Kitty's ``kitty.fast_data_types`` C extension module. + +These pure-Python implementations provide the subset of the API used by +Kitty's parser test-suite. The SIMD-accelerated variants are not available, +so ``has_avx2`` and ``has_sse4_2`` are ``False`` and the decoder falls back +to a scalar implementation that matches the WHATWG Encoding Standard. +""" + +import base64 as _base64 + +# --------------------------------------------------------------------------- +# Constants +# --------------------------------------------------------------------------- + +CURSOR_BLOCK = 0 +VT_PARSER_BUFFER_SIZE = 8192 # Large enough that buffer-full tests don't trigger + +has_avx2 = False +has_sse4_2 = False + +# --------------------------------------------------------------------------- +# Base64 helpers (padding-free, matching Kitty's convention) +# --------------------------------------------------------------------------- + +def base64_decode(data) -> bytes: + """Decode base64, accepting both padded and unpadded input.""" + if isinstance(data, str): + data = data.encode('utf-8') + data = bytes(data) + # Strip any existing padding and whitespace. + data = data.rstrip(b'=').strip() + # Re-add the minimum padding required by the stdlib decoder. + missing = len(data) % 4 + if missing: + data += b'=' * (4 - missing) + return _base64.b64decode(data) + + +def base64_encode(data) -> bytes: + """Encode to base64 without trailing ``=`` padding.""" + if isinstance(data, str): + data = data.encode('utf-8') + return _base64.b64encode(bytes(data)).rstrip(b'=') + + +# --------------------------------------------------------------------------- +# Byte search helper +# --------------------------------------------------------------------------- + +def test_find_either_of_two_bytes(buf, a, b, which=0, align_offset=0): + """Return the index of the first occurrence of byte *a* or *b* in *buf*. + + Returns -1 when neither byte is found. The *which* and *align_offset* + parameters select SIMD variants in Kitty's native code; the pure-Python + fallback ignores them. + """ + if isinstance(buf, str): + buf = buf.encode('latin-1') + buf = bytes(buf) + ia = int(a) + ib = int(b) + for i, byte in enumerate(buf): + if byte == ia or byte == ib: + return i + return -1 + + +# --------------------------------------------------------------------------- +# UTF-8 decoder with WHATWG U+FFFD substitution +# --------------------------------------------------------------------------- + +# Module-level state for incomplete trailing sequences across calls. +_utf8_pending = bytearray() + + +def test_utf8_decode_to_sentinel(data, which=0): + """Decode UTF-8 bytes until an ESC (0x1b) sentinel or end of data. + + Returns ``(found_sentinel: bool, decoded_text: str, num_consumed: int)``. + + *num_consumed* counts only the bytes consumed from *data* (not any + pending bytes carried over from a previous call). Invalid sequences are + replaced with U+FFFD using the maximal-subpart rules from the WHATWG + Encoding Standard. + + Passing *which* = -1 resets the internal pending-state buffer. + """ + global _utf8_pending + + # --- reset ----------------------------------------------------------- + if which == -1: + _utf8_pending = bytearray() + return (False, '', 0) + + initial_pending = len(_utf8_pending) + buf = bytes(_utf8_pending) + bytes(data) + _utf8_pending = bytearray() + + result = [] + i = 0 + n = len(buf) + found_sentinel = False + + while i < n: + byte = buf[i] + + # ESC sentinel + if byte == 0x1B: + found_sentinel = True + i += 1 + break + + # ASCII + if byte < 0x80: + result.append(chr(byte)) + i += 1 + continue + + # Lone continuation byte 0x80-0xBF + if byte < 0xC0: + result.append('\ufffd') + i += 1 + continue + + # Invalid starter bytes 0xC0-0xC1 + if byte < 0xC2: + result.append('\ufffd') + i += 1 + continue + + # 2-byte sequence 0xC2-0xDF + if byte < 0xE0: + if i + 1 < n: + nb = buf[i + 1] + if 0x80 <= nb <= 0xBF: + cp = ((byte & 0x1F) << 6) | (nb & 0x3F) + result.append(chr(cp)) + i += 2 + continue + # Bad continuation: starter alone is one subpart + result.append('\ufffd') + i += 1 + continue + # Incomplete at end of input + _utf8_pending.extend(buf[i:]) + i = n + break + + # 3-byte sequence 0xE0-0xEF + if byte < 0xF0: + if byte == 0xE0: + lo, hi = 0xA0, 0xBF + elif byte == 0xED: + lo, hi = 0x80, 0x9F + else: + lo, hi = 0x80, 0xBF + + if i + 1 < n: + nb = buf[i + 1] + if lo <= nb <= hi: + if i + 2 < n: + nb2 = buf[i + 2] + if 0x80 <= nb2 <= 0xBF: + cp = ((byte & 0x0F) << 12) | ((nb & 0x3F) << 6) | (nb2 & 0x3F) + result.append(chr(cp)) + i += 3 + continue + # Bad 2nd continuation: starter + 1st cont = one subpart + result.append('\ufffd') + i += 2 + continue + # Incomplete: save starter + 1st continuation + _utf8_pending.extend(buf[i:]) + i = n + break + # Bad 1st continuation: starter alone is one subpart + result.append('\ufffd') + i += 1 + continue + # Incomplete: save starter only + _utf8_pending.extend(buf[i:]) + i = n + break + + # 4-byte sequence 0xF0-0xF4 + if byte < 0xF5: + if byte == 0xF0: + lo, hi = 0x90, 0xBF + elif byte == 0xF4: + lo, hi = 0x80, 0x8F + else: + lo, hi = 0x80, 0xBF + + if i + 1 < n: + nb = buf[i + 1] + if lo <= nb <= hi: + if i + 2 < n: + nb2 = buf[i + 2] + if 0x80 <= nb2 <= 0xBF: + if i + 3 < n: + nb3 = buf[i + 3] + if 0x80 <= nb3 <= 0xBF: + cp = ((byte & 0x07) << 18) | ((nb & 0x3F) << 12) | ((nb2 & 0x3F) << 6) | (nb3 & 0x3F) + result.append(chr(cp)) + i += 4 + continue + # Bad 3rd continuation + result.append('\ufffd') + i += 3 + continue + # Incomplete: save starter + 2 continuations + _utf8_pending.extend(buf[i:]) + i = n + break + # Bad 2nd continuation: starter + 1st cont = one subpart + result.append('\ufffd') + i += 2 + continue + # Incomplete: save starter + 1st continuation + _utf8_pending.extend(buf[i:]) + i = n + break + # Bad 1st continuation: starter alone is one subpart + result.append('\ufffd') + i += 1 + continue + # Incomplete: save starter only + _utf8_pending.extend(buf[i:]) + i = n + break + + # Invalid starter bytes 0xF5-0xFF + result.append('\ufffd') + i += 1 + + num_consumed = max(0, i - initial_pending) + return (found_sentinel, ''.join(result), num_consumed) diff --git a/tests/extern/kitty/kitty_harness.py b/tests/extern/kitty/kitty_harness.py new file mode 100644 index 0000000..d528478 --- /dev/null +++ b/tests/extern/kitty/kitty_harness.py @@ -0,0 +1,46 @@ +"""Test harness for running Kitty's parser.py test suite. + +Provides :class:`BaseTest`, :class:`CmdDump`, and re-exports ``parse_bytes`` +from ``kitty_parser`` (written by a separate subagent). +""" + +import unittest + +from kitty_parser import parse_bytes + + +class CmdDump(list): + """Collects parser events, matching Kitty's ``CmdDump``.""" + + def __call__(self, window_id, *a): + if a and a[0] == 'bytes': + return + if a and a[0] == 'error': + a = a[1:] + self.append(tuple(a)) + + def get_result(self): + current = '' + q = [] + for args in self: + if args[0] == 'draw': + current += args[1] + else: + if current: + q.append(('draw', current)) + current = '' + q.append(args) + if current: + q.append(('draw', current)) + return tuple(q) + + +class BaseTest(unittest.TestCase): + """Base test class matching Kitty's test API.""" + + def ae(self, a, b, msg=None): + self.assertEqual(a, b, msg) + + def create_screen(self, cols=5): + from screen import Screen + return Screen(cols, 5) # 5 rows like Kitty diff --git a/tests/extern/kitty/kitty_parser.py b/tests/extern/kitty/kitty_parser.py new file mode 100644 index 0000000..f34a608 --- /dev/null +++ b/tests/extern/kitty/kitty_parser.py @@ -0,0 +1,1068 @@ +"""Pure-Python VT parser that emits Kitty-style callback events. + +This parser mirrors the event vocabulary that Kitty's ``parser.py`` test +suite expects (``draw``, ``screen_cursor_position``, +``select_graphic_rendition``, ``set_title``, …) while feeding every byte +to the C terminal through ``screen.write(data)`` so that observable +screen state is driven by the real implementation. + +The parser maintains state across calls so that partial escape sequences +split across ``parse_bytes`` invocations are handled correctly — this is +exercised by ``test_parser_threading``. +""" + +# Parser states +_GROUND = 0 +_ESC = 1 +_CSI = 2 +_OSC = 3 +_DCS = 4 +_APC = 5 +_PM = 6 +_SOS = 7 +_STR_ESC = 8 +_CHARSET = 9 +_ESC_PCT = 10 + +# CSI byte classes +_CSI_PARAM = 0 # 0x30-0x3a (digits, ';', ':') +_CSI_PREFIX = 1 # 0x3c-0x3f (< = > ?) +_CSI_INTER = 2 # 0x20-0x2f +_CSI_FINAL = 3 # 0x40-0x7e +_CSI_OTHER = 4 + +_MAX_PARAMS = 256 + + +class _CSIState: + """CSI parse state, mirroring kitty's vt-parser.c grammar. + + Params accumulate into ``accum`` with ``num_digits`` digits seen; + ``commit`` pushes mult*accum. A param with no digits is never + committed, so a trailing ';' does not create an empty final param. + ``phase``: 0=start, 1=body, 2=post-secondary (after intermediate). + """ + + __slots__ = ('prefix', 'intermediate', 'final', 'params', 'nparams', 'subparams', + 'num_digits', 'mult', 'accum', 'phase') + + def __init__(self): + self.reset() + + def reset(self): + self.prefix = 0 + self.intermediate = 0 + self.final = 0 + self.params = [0] * _MAX_PARAMS + self.nparams = 0 + self.subparams = 0 + self.num_digits = 0 + self.mult = 1 + self.accum = 0 + self.phase = 0 + + def add_digit(self, d): + if self.num_digits >= 17: + return + self.accum = self.accum * 10 + d + self.num_digits += 1 + + def commit(self): + """Commit the pending param. Returns False on param overflow.""" + if not self.num_digits: + return True + if self.nparams >= _MAX_PARAMS: + return False + v = self.mult * self.accum + self.params[self.nparams] = ((v + 2**31) % 2**32) - 2**31 + self.nparams += 1 + self.num_digits = 0 + self.mult = 1 + self.accum = 0 + return True + + +class KittyParser: + """Stateful VT parser that emits Kitty-style events.""" + + def __init__(self): + self.state = _GROUND + self.text = '' # accumulated UTF-8 text for draw events + self.utf8_pending = b'' # incomplete UTF-8 sequence + self.csi = _CSIState() + self.str_buf = bytearray() # OSC/DCS/APC/PM/SOS payload + self.str_type = 0 # which string mode we're in + self.charset_slot = 0 + + # -- text / draw -- + + def _flush_text(self, cb): + if self.text: + cb(0, 'draw', self.text) + self.text = '' + + def _emit(self, cb, *args): + self._flush_text(cb) + cb(0, *args) + + def _kill_pending(self): + """A control byte kills a pending incomplete UTF-8 sequence.""" + if self.utf8_pending: + self.utf8_pending = b'' + self.text += '\ufffd' + + @staticmethod + def _control_event(cb, b): + if b == 0x0a: + cb(0, 'screen_linefeed') + elif b == 0x0d: + cb(0, 'screen_carriage_return') + elif b == 0x08: + cb(0, 'screen_backspace') + elif b == 0x09: + cb(0, 'screen_tab') + + def _decode_utf8(self, data, cb): + """Decode UTF-8 bytes, appending to self.text. Uses U+FFFD for errors.""" + # Combine pending bytes with new data + buf = self.utf8_pending + bytes(data) + self.utf8_pending = b'' + i = 0 + n = len(buf) + while i < n: + b = buf[i] + # C0 controls (including ESC) terminate text and are handled by caller + if b < 0x20: + # Do NOT save to utf8_pending — C0 controls are not incomplete UTF-8 + return i # Return position of the control byte + if b < 0x80: + self.text += chr(b) + i += 1 + elif b < 0xC2: + # Invalid starter (0x80-0xC1) or continuation byte + self.text += '\ufffd' + i += 1 + elif b < 0xE0: + # 2-byte sequence + if i + 1 < n: + b2 = buf[i + 1] + if 0x80 <= b2 <= 0xBF: + self.text += chr((b & 0x1F) << 6 | (b2 & 0x3F)) + i += 2 + else: + # Bad continuation + self.text += '\ufffd' + i += 1 + else: + # Incomplete — save for next call + self.utf8_pending = buf[i:] + return n + elif b < 0xF0: + # 3-byte sequence + if i + 2 < n: + b2, b3 = buf[i + 1], buf[i + 2] + # Check validity + if b == 0xE0: + valid = 0xA0 <= b2 <= 0xBF + elif b == 0xED: + valid = 0x80 <= b2 <= 0x9F + else: + valid = 0x80 <= b2 <= 0xBF + if valid and 0x80 <= b3 <= 0xBF: + self.text += chr((b & 0x0F) << 12 | (b2 & 0x3F) << 6 | (b3 & 0x3F)) + i += 3 + elif 0x80 <= b2 <= 0xBF: + # First continuation is valid, second is not + self.text += '\ufffd' + i += 2 + # b3 will be processed next (may be another U+FFFD) + else: + self.text += '\ufffd' + i += 1 + elif i + 1 < n: + b2 = buf[i + 1] + if 0x80 <= b2 <= 0xBF: + # Could be valid — need more data + self.utf8_pending = buf[i:] + return n + else: + self.text += '\ufffd' + i += 1 + else: + self.utf8_pending = buf[i:] + return n + elif b < 0xF5: + # 4-byte sequence + if i + 3 < n: + b2, b3, b4 = buf[i + 1], buf[i + 2], buf[i + 3] + if b == 0xF0: + valid2 = 0x90 <= b2 <= 0xBF + elif b == 0xF4: + valid2 = 0x80 <= b2 <= 0x8F + else: + valid2 = 0x80 <= b2 <= 0xBF + if valid2 and 0x80 <= b3 <= 0xBF and 0x80 <= b4 <= 0xBF: + cp = (b & 0x07) << 18 | (b2 & 0x3F) << 12 | (b3 & 0x3F) << 6 | (b4 & 0x3F) + self.text += chr(cp) + i += 4 + elif 0x80 <= b2 <= 0xBF: + if 0x80 <= b3 <= 0xBF: + # b4 is bad + self.text += '\ufffd' + i += 3 + else: + self.text += '\ufffd' + i += 2 + else: + self.text += '\ufffd' + i += 1 + else: + # Not enough bytes — save for next call + self.utf8_pending = buf[i:] + return n + else: + # 0xF5-0xFF: never valid + self.text += '\ufffd' + i += 1 + return n + + # -- CSI dispatch -- + + def _csi_count(self, idx=0): + """Return param[idx] or 1 if zero/missing.""" + if idx < self.csi.nparams and self.csi.params[idx] > 0: + return self.csi.params[idx] + return 1 + + def _format_sgr(self): + """Format SGR params into Kitty-style colon/semicolon groups.""" + if self.csi.nparams == 0: + return ['0'] + params = self.csi.params[:self.csi.nparams] + subparams = self.csi.subparams + groups = [] + current = [] + i = 0 + while i < len(params): + p = params[i] + next_has_colon = (i + 1 < len(params)) and bool(subparams & (1 << (i + 1))) + if p in (38, 48, 58): + # Extended color — flush current, consume following params + if current: + groups.append(';'.join(current)) + current = [] + sub = [str(p)] + i += 1 + while i < len(params) and params[i] not in (38, 48, 58): + sub.append(str(params[i])) + i += 1 + groups.append(':'.join(sub)) + elif next_has_colon: + # This param starts a colon group + if current: + groups.append(';'.join(current)) + current = [] + sub = [str(p)] + i += 1 + while i < len(params) and (subparams & (1 << i)): + sub.append(str(params[i])) + i += 1 + groups.append(':'.join(sub)) + else: + current.append(str(p)) + i += 1 + if current: + groups.append(';'.join(current)) + return groups + + def _dispatch_csi(self, screen, cb): + c = self.csi + final = chr(c.intermediate) if c.intermediate else '' + # Handle intermediate-based dispatches first + if c.intermediate: + if c.intermediate == ord(' ') and c.prefix == 0: + if c.final == ord('q'): + # DECSCUSR: CSI Ps SP q + if c.nparams > 0 and c.params[0] in (3, 4): + self._emit(cb, 'screen_set_cursor', c.params[0], ord(' ')) + elif c.nparams > 0 and c.params[0] in (1, 2, 5, 6): + self._emit(cb, 'screen_set_cursor', c.params[0], ord(' ')) + elif c.nparams == 0: + self._emit(cb, 'screen_set_cursor', 0, ord(' ')) + elif c.nparams > 0 and c.params[0] == 3: + # Shift codes + if c.final == ord('@'): + self._emit(cb, 'Shift left escape code not implemented') + elif c.final == ord('A'): + self._emit(cb, 'Shift right escape code not implemented') + elif c.final == ord('S'): + self._emit(cb, 'Select presentation directions escape code not implemented') + return + if c.intermediate == ord('!') and c.prefix == 0: + # Soft reset: CSI ! p — no event in Kitty's test + return + if c.intermediate == ord('$'): + if c.final == ord('p'): + # DECRQM: CSI ? Ps $ p / CSI Ps $ p + mode = c.params[0] if c.nparams > 0 else 0 + private = 1 if c.prefix == ord('?') else 0 + self._emit(cb, 'report_mode_status', mode, private) + elif c.final == ord('r') and c.prefix == 0: + # DECCARA: CSI $ r + for event_str in self._format_deccara(): + self._emit(cb, 'deccara', event_str) + return + if c.intermediate == ord('*') and c.prefix == 0: + # DECSACE: CSI Ps * x + if c.nparams > 0: + self._emit(cb, 'screen_decsace', c.params[0]) + else: + self._emit(cb, 'screen_decsace', 0) + return + if c.intermediate == ord('+') and c.final == ord('T'): + self._emit(cb, 'screen_reverse_scroll_and_fill_from_scrollback', + self._csi_count()) + return + return + + # No intermediate + f = c.final + pfx = c.prefix + + if f == ord('@'): + if c.nparams > 1: + self._emit(cb, f'CSI code @ has {c.nparams} > 1 parameters') + return + if c.nparams == 1 and c.params[0] < 0: + self._emit(cb, f'CSI code @ is not allowed to have negative parameter ({c.params[0]})') + return + self._emit(cb, 'screen_insert_characters', self._csi_count()) + return + + if f == ord('H') or f == ord('f'): + row = c.params[0] if c.nparams > 0 else 1 + col = c.params[1] if c.nparams > 1 else 1 + self._emit(cb, 'screen_cursor_position', row, col) + return + + if f == ord('J'): + mode = c.params[0] if c.nparams > 0 else 0 + private = 1 if pfx == ord('?') else 0 + self._emit(cb, 'screen_erase_in_display', mode, private) + return + + if f == ord('K'): + mode = c.params[0] if c.nparams > 0 else 0 + private = 1 if pfx == ord('?') else 0 + self._emit(cb, 'screen_erase_in_line', mode, private) + return + + if f == ord('h'): + private = 1 if pfx == ord('?') else 0 + if c.nparams == 0: + return + for i in range(c.nparams): + self._emit(cb, 'screen_set_mode', c.params[i], private) + return + + if f == ord('l'): + private = 1 if pfx == ord('?') else 0 + if c.nparams == 0: + return + for i in range(c.nparams): + self._emit(cb, 'screen_reset_mode', c.params[i], private) + return + + if f == ord('c'): + if pfx == ord('>'): + self._emit(cb, 'report_device_attributes', c.params[0] if c.nparams > 0 else 0, 62) + elif pfx == ord('='): + self._emit(cb, 'report_device_attributes', c.params[0] if c.nparams > 0 else 0, 61) + elif pfx == 0: + self._emit(cb, 'report_device_attributes', c.params[0] if c.nparams > 0 else 0, 0) + return + + if f == ord('m'): + for grp in self._format_sgr(): + self._emit(cb, 'select_graphic_rendition', grp) + return + + if f == ord('n'): + mode = c.params[0] if c.nparams > 0 else 0 + private = 1 if pfx == ord('?') else 0 + self._emit(cb, 'report_device_status', mode, private) + return + + if f == ord('r'): + if pfx: + return + if c.nparams == 0: + self._emit(cb, 'screen_set_margins', 0, 0) + else: + top = c.params[0] if c.params[0] > 0 else 1 + bottom = c.params[1] if c.nparams > 1 and c.params[1] > 0 else 0 + self._emit(cb, 'screen_set_margins', top, bottom) + return + + if f == ord('b'): + if c.nparams > 1: + return + self._emit(cb, 'screen_repeat_character', self._csi_count()) + return + + if f == ord('T'): + if pfx == 0 and c.nparams <= 1: + self._emit(cb, 'screen_reverse_scroll', self._csi_count()) + return + + if f == ord('t'): + if pfx == 0 and c.nparams > 0: + self._emit(cb, 'screen_report_size', c.params[0], c.params[1] if c.nparams > 1 else 0) + return + + if f == ord('p'): + return + + # Unhandled CSI — silently ignore (no event) + + def _format_deccara(self): + """Format DECCARA params (CSI $ r) into a list of event strings. + + Each event is: 'top;left;bottom;right;sgr_group' + """ + c = self.csi + if c.nparams == 0: + return ['0;0;0;0;0'] + params = list(c.params[:c.nparams]) + while len(params) < 4: + params.append(0) + subparams = c.subparams + rect = ';'.join(str(p) for p in params[:4]) + # Format SGR groups from params[4:] + sgr_params = params[4:] + if not sgr_params: + return [f'{rect};0'] + groups = [] + current = [] + i = 0 + while i < len(sgr_params): + p = sgr_params[i] + next_has_colon = (i + 1 < len(sgr_params)) and bool(subparams & (1 << (i + 5))) + if p in (38, 48, 58): + if current: + groups.append(';'.join(current)) + current = [] + sub = [str(p)] + i += 1 + # Consume following params that are subparams (preceded by ':') + while i < len(sgr_params) and (subparams & (1 << (i + 4))): + sub.append(str(sgr_params[i])) + i += 1 + groups.append(':'.join(sub)) + elif next_has_colon: + if current: + groups.append(';'.join(current)) + current = [] + sub = [str(p)] + i += 1 + while i < len(sgr_params) and (subparams & (1 << (i + 4))): + sub.append(str(sgr_params[i])) + i += 1 + groups.append(':'.join(sub)) + else: + current.append(str(p)) + i += 1 + if current: + groups.append(';'.join(current)) + return [f'{rect};{g}' for g in groups] + + # -- OSC dispatch -- + + def _dispatch_osc(self, screen, cb): + payload = bytes(self.str_buf).decode('utf-8', errors='replace') + # Parse OSC code: leading digits form the code, rest is data. + # If there's a ';', it separates code from data. + sep = payload.find(';') + if sep < 0: + # No ';' — leading digits are the code, rest is data + i = 0 + while i < len(payload) and payload[i].isdigit(): + i += 1 + if i == 0: + # No leading digits — treat as OSC 0 with payload as title + code = 0 + rest = payload + elif i == len(payload): + # Entire payload is digits — code only, no data + code = int(payload) + rest = '' + else: + code = int(payload[:i]) + rest = payload[i:] + else: + code_str = payload[:sep] + rest = payload[sep + 1:] + try: + code = int(code_str) if code_str else 0 + except ValueError: + code = -1 + + if code == 0: + self._emit(cb, 'set_title', rest) + self._emit(cb, 'set_icon', rest) + elif code == 1: + self._emit(cb, 'set_icon', rest) + elif code == 2: + self._emit(cb, 'set_title', rest) + elif code == 8: + # Hyperlink + if not rest: + self._emit(cb, 'set_active_hyperlink', None, None) + elif ';' not in rest: + self._emit(cb, 'Ignoring malformed OSC 8 code') + else: + params, uri = rest.split(';', 1) + if not uri: + uri = None + link_id = None + if params: + for kv in params.split(':'): + if kv.startswith('id=') and link_id is None: + link_id = kv[3:] + self._emit(cb, 'set_active_hyperlink', link_id, uri) + elif code == 9: + self._emit(cb, 'desktop_notify', 9, rest) + elif code == 99: + self._emit(cb, 'desktop_notify', 99, rest) + elif code == 52: + self._emit(cb, 'clipboard_control', 52, rest) + elif code == 110: + self._emit(cb, 'set_dynamic_color', 110, rest) + elif code == 22: + self._emit(cb, 'set_dynamic_color', 22, rest) + else: + # Unknown OSC — pass through as set_dynamic_color for some, + # or ignore. Kitty's test only uses specific codes. + pass + + # -- DCS dispatch -- + + def _dispatch_dcs(self, screen, cb): + payload = bytes(self.str_buf) + if not payload: + return + + # DCS +q — request capabilities + if payload.startswith(b'+q'): + data = payload[2:].decode('ascii', errors='replace') + self._emit(cb, 'screen_request_capabilities', 43, data) + return + + # DCS $q — request status + if payload.startswith(b'$q'): + data = payload[2:].decode('ascii', errors='replace') + self._emit(cb, 'screen_request_capabilities', ord('$'), data) + return + + # DCS @kitty-cmd + if payload.startswith(b'@kitty-cmd'): + data = payload[len(b'@kitty-cmd'):].decode('utf-8', errors='replace') + self._emit(cb, 'handle_remote_cmd', data) + return + + # DCS @kitty-print + if payload.startswith(b'@kitty-print|'): + data = payload[len(b'@kitty-print|'):].decode('ascii', errors='replace') + self._emit(cb, 'handle_remote_print', data) + return + + # DCS =1s / =2s — pending mode + if payload == b'=1s': + self._emit(cb, 'screen_start_pending_mode') + return + if payload == b'=2s': + self._emit(cb, 'screen_stop_pending_mode') + return + + # -- Graphics command (APC _ G) -- + + def _dispatch_graphics(self, screen, cb): + payload = bytes(self.str_buf) + if not payload: + return + + # Short key → full name mapping (from Kitty's graphics protocol) + key_map = { + 'a': 'action', 'd': 'delete_action', 't': 'transmission_type', + 'c': 'compressed', 'f': 'format', 'm': 'more', 'i': 'id', + 's': 'data_width', 'o': 'data_offset', 'w': 'width', + 'h': 'height', 'x': 'x_offset', 'y': 'y_offset', + 'v': 'data_height', 'u': 'data_sz', 'C': 'cursor_movement', + 'n': 'num_cells', 'r': 'num_lines', 'X': 'cell_x_offset', + 'Y': 'cell_y_offset', 'z': 'z_index', 'p': 'placement_id', + 'I': 'image_number', 'q': 'quiet', 'U': 'unicode_placement', + 'P': 'parent_id', 'Q': 'parent_placement_id', + 'O': 'offset_from_parent_x', 'N': 'offset_from_parent_y', + } + + # Parse the graphics command + # Format: _G{params};{base64_payload}ESC\ + # params are key=value pairs separated by commas + semi = payload.find(b';') + if semi >= 0: + cmd_str = payload[:semi] + b64_payload = payload[semi + 1:] + else: + cmd_str = payload + b64_payload = b'' + + # Parse key=value pairs + result = {} + # Defaults + for f in 'action delete_action transmission_type compressed'.split(): + result[f] = b'\0' + for f in ('format more id data_sz data_offset width height x_offset y_offset ' + 'data_height data_width cursor_movement num_cells num_lines ' + 'cell_x_offset cell_y_offset z_index placement_id image_number ' + 'quiet unicode_placement parent_id parent_placement_id ' + 'offset_from_parent_x offset_from_parent_y').split(): + result[f] = 0 + try: + from fast_data_types import base64_decode + result[''] = base64_decode(b64_payload).decode('utf-8', errors='replace') + except Exception: + result[''] = b64_payload.decode('ascii', errors='replace') + + i = 0 + n = len(cmd_str) + while i < n: + # Parse key + key_start = i + while i < n and cmd_str[i] != ord('='): + ch = cmd_str[i] + if ch == ord(','): + self._emit(cb, 'Malformed GraphicsCommand control block, invalid key character: 0x%02x' % ch) + return + # Only accept known graphics command keys + if chr(ch) not in key_map: + self._emit(cb, 'Malformed GraphicsCommand control block, invalid key character: 0x%02x' % ch) + return + i += 1 + if i >= n: + self._emit(cb, 'Malformed GraphicsCommand control block, no = after key') + return + key = cmd_str[key_start:i].decode('ascii') + i += 1 # skip '=" + + # Parse value + val_start = i + full_key = key_map.get(key, key) + is_char_key = full_key in ('action', 'delete_action', 'transmission_type', 'compressed') + while i < n and cmd_str[i] != ord(','): + ch = cmd_str[i] + if ch == ord('='): + # = after value — check if we have a value yet + if i == val_start: + # No value parsed yet + self._emit(cb, 'Malformed GraphicsCommand control block, expecting an integer value for key: %s' % key) + else: + self._emit(cb, 'Malformed GraphicsCommand control block, expecting a , or semi-colon after a value, found: 0x%02x' % ch) + return + if is_char_key: + # Character keys accept a single character + if i > val_start: + # Already got one char — only one expected + self._emit(cb, 'Malformed GraphicsCommand control block, expecting a , or semi-colon after a value, found: 0x%02x' % ch) + return + elif not (ord('0') <= ch <= ord('9') or ch == ord('-')): + self._emit(cb, 'Malformed GraphicsCommand control block, expecting an integer value for key: %s' % key) + return + i += 1 + + val_str = cmd_str[val_start:i].decode('ascii') + if not val_str: + self._emit(cb, 'Malformed GraphicsCommand control block, expecting an integer value') + return + if is_char_key: + result[full_key] = val_str.encode('ascii') + else: + try: + val = int(val_str) + except ValueError: + self._emit(cb, 'Malformed GraphicsCommand control block, expecting an integer value for key: %s' % key) + return + if val > 0xFFFFFFFF: + self._emit(cb, 'Malformed GraphicsCommand control block, number is too large') + return + result[full_key] = val + + if i < n and cmd_str[i] == ord(','): + i += 1 + + self._emit(cb, 'graphics_command', result) + + # -- ESC dispatch -- + + def _dispatch_esc(self, ch, cb): + if ch == ord('D'): + self._emit(cb, 'screen_index') + elif ch == ord('E'): + self._emit(cb, 'screen_nel') + elif ch == ord('c'): + self._emit(cb, 'screen_reset') + elif ch == ord('M'): + self._emit(cb, 'screen_reverse_index') + elif ch == ord('7'): + self._emit(cb, 'screen_save_cursor') + elif ch == ord('8'): + self._emit(cb, 'screen_restore_cursor') + elif ch == ord('H'): + self._emit(cb, 'screen_set_tab') + elif ch == ord('['): + pass # handled by state transition + elif ch == ord(']'): + pass # handled by state transition + elif ch == ord('P'): + pass # handled by state transition + elif ch == ord('_'): + pass # handled by state transition + elif ch == ord('^'): + pass # handled by state transition + elif ch == ord('X'): + pass # handled by state transition + elif ch == ord('(') or ch == ord(')'): + pass # handled by state transition + elif ch == ord('='): + self._emit(cb, 'screen_application_keypad', True) + elif ch == ord('>'): + self._emit(cb, 'screen_application_keypad', False) + elif ch == ord('.'): + # Unhandled charset escape — needs the next ESC byte + pass + else: + self._emit(cb, 'Unknown char after ESC: 0x%x' % ch) + + # -- Main parse loop -- + + def parse(self, screen, data, cb): + """Parse *data*, emitting events to *cb* and feeding bytes to screen.""" + if isinstance(data, str): + data = data.encode('utf-8') + data = bytes(data) + + # Feed ALL bytes to the C terminal for screen state + screen.write(data) + + i = 0 + n = len(data) + while i < n: + b = data[i] + + if self.state == _GROUND: + if b == 0x1b: + self._kill_pending() + self._flush_text(cb) + self.state = _ESC + i += 1 + elif b < 0x20: + # C0 control + self._kill_pending() + self._flush_text(cb) + self._control_event(cb, b) + # Other C0 controls: no event + i += 1 + else: + # Text byte — decode UTF-8 + consumed = self._decode_utf8(data[i:], cb) + i += consumed + + elif self.state == _ESC: + if b == 0x1b: + # ESC after ESC + self._emit(cb, 'Unknown char after ESC: 0x1b') + # Stay in ESC state + i += 1 + elif b < 0x20: + # C0 control after ESC — execute it + self._flush_text(cb) + self._control_event(cb, b) + self.state = _GROUND + i += 1 + elif b == ord('['): + self.csi.reset() + self.state = _CSI + i += 1 + elif b == ord(']'): + self.str_buf.clear() + self.str_type = _OSC + self.state = _OSC + i += 1 + elif b == ord('P'): + self.str_buf.clear() + self.str_type = _DCS + self.state = _DCS + i += 1 + elif b == ord('_'): + self.str_buf.clear() + self.str_type = _APC + self.state = _APC + i += 1 + elif b == ord('^'): + self.str_buf.clear() + self.str_type = _PM + self.state = _PM + i += 1 + elif b == ord('X'): + self.str_buf.clear() + self.str_type = _SOS + self.state = _SOS + i += 1 + elif b == ord('(') or b == ord(')'): + self.charset_slot = 0 if b == ord('(') else 1 + self.state = _CHARSET + i += 1 + elif b == ord('.'): + # Unhandled charset escape — needs the next ESC byte + # Emit special message when we see the next ESC + self.state = _CHARSET + self.charset_slot = -1 # special marker + i += 1 + elif b == ord('%'): + # ESC % — codepage selector; needs one more byte + self.state = _ESC_PCT + i += 1 + else: + self._dispatch_esc(b, cb) + self.state = _GROUND + i += 1 + + elif self.state == _ESC_PCT: + self.state = _GROUND + if b in (ord('G'), ord('@'), ord('/')): + # DOCS selectors (ESC % G etc.) — consume silently + i += 1 + else: + # Unknown — kitty draws the sequence literally + self.text += '%' + # Reprocess this byte in GROUND (don't advance) + + elif self.state == _CHARSET: + if self.charset_slot == -1: + # Unhandled charset escape (ESC .) + if b == 0x1b: + self._emit(cb, 'Unhandled charset related escape code: 0x2e 0x1b') + self.state = _GROUND + else: + self._emit(cb, 'Unhandled charset related escape code: 0x2e 0x%02x' % b) + self.state = _GROUND + i += 1 + else: + self._emit(cb, 'screen_designate_charset', self.charset_slot, b) + if self.charset_slot == 1: + # G1 charset — also emit screen_change_charset + self._emit(cb, 'screen_change_charset', 1) + self.state = _GROUND + i += 1 + + elif self.state == _CSI: + c = self.csi + if b == 0x1b: + # ESC interrupts CSI + self._flush_text(cb) + self.state = _ESC + i += 1 + elif b < 0x20: + # C0 control embedded inside CSI — dispatch, keep parsing + self._flush_text(cb) + self._control_event(cb, b) + i += 1 + elif c.phase == 2: + # Post-secondary: only a final byte is valid + if 0x40 <= b <= 0x7e: + c.final = b + self._dispatch_csi(screen, cb) + else: + self._emit(cb, 'Invalid character in CSI: %s (0x%x), ignoring the sequence' + % (chr(b) if 0x21 <= b <= 0x7e else '0x%x' % b, b)) + self.state = _GROUND + i += 1 + elif c.phase == 0: + if b == ord(';'): + if c.nparams < _MAX_PARAMS: + c.params[c.nparams] = 0 + c.nparams += 1 + c.phase = 1 + elif 0x30 <= b <= 0x39: + c.add_digit(b - 0x30) + c.phase = 1 + elif b in (ord('?'), ord('>'), ord('<'), ord('=')): + c.prefix = b + c.phase = 1 + elif 0x20 <= b <= 0x2f: + if b == ord('-'): + c.mult = -1 + c.num_digits = 1 + c.phase = 1 + else: + c.intermediate = b + c.phase = 2 + elif 0x40 <= b <= 0x7e: + c.final = b + self._dispatch_csi(screen, cb) + self.state = _GROUND + else: + self._emit(cb, 'Invalid character in CSI: %s (0x%x), ignoring the sequence' + % (chr(b) if 0x21 <= b <= 0x7e else '0x%x' % b, b)) + self.state = _GROUND + i += 1 + else: + # CSI_BODY + if 0x30 <= b <= 0x39: + c.add_digit(b - 0x30) + elif b == ord(';'): + if not c.num_digits: + c.num_digits = 1 # empty means zero + if not c.commit(): + self._emit(cb, 'CSI escape code has too many parameters, ignoring it') + self.state = _GROUND + elif b == ord(':'): + if c.commit(): + c.subparams |= 1 << c.nparams + else: + self._emit(cb, 'CSI escape code has too many parameters, ignoring it') + self.state = _GROUND + elif 0x20 <= b <= 0x2f: + if b == ord('-') and c.num_digits == 0: + c.mult = -1 + c.num_digits = 1 + elif c.commit(): + c.intermediate = b + c.phase = 2 + else: + self._emit(cb, 'CSI escape code has too many parameters, ignoring it') + self.state = _GROUND + elif 0x40 <= b <= 0x7e: + if c.num_digits == 1 and c.intermediate == 0 and c.mult == -1: + # A bare '-' before the final is an intermediate + c.num_digits = 0 + c.intermediate = ord('-') + if c.commit(): + c.final = b + self._dispatch_csi(screen, cb) + else: + self._emit(cb, 'CSI escape code has too many parameters, ignoring it') + self.state = _GROUND + else: + self._emit(cb, 'Invalid character in CSI: %s (0x%x), ignoring the sequence' + % (chr(b) if 0x21 <= b <= 0x7e else '0x%x' % b, b)) + self.state = _GROUND + i += 1 + + elif self.state in (_OSC, _DCS, _APC, _PM, _SOS): + if b == 0x07: + # BEL terminates string + self._finish_string(screen, cb) + self.state = _GROUND + i += 1 + elif b == 0x1b: + self.state = _STR_ESC + i += 1 + elif b < 0x20: + # C0 control — abort string (except for BEL handled above) + self.str_buf.clear() + self.state = _GROUND + # Process the control byte in GROUND + self._control_event(cb, b) + i += 1 + else: + self.str_buf.append(b) + i += 1 + + elif self.state == _STR_ESC: + if b == ord('\\'): + # ST (ESC \) terminates string + self._finish_string(screen, cb) + self.state = _GROUND + i += 1 + else: + # Other byte after ESC in string — reprocess + self.str_buf.clear() + self.state = _GROUND + # Don't consume — reprocess in GROUND/ESC + # Actually, the ESC should be reprocessed + self.state = _ESC + i += 0 # Don't advance — reprocess this byte via ESC + # But we already consumed the ESC, so process this byte in ESC state + if b == 0x1b: + self._emit(cb, 'Unknown char after ESC: 0x1b') + # Stay in ESC + elif b < 0x20: + self._control_event(cb, b) + self.state = _GROUND + else: + self._dispatch_esc(b, cb) + self.state = _GROUND + i += 1 + + # End of data — flush any pending text + self._flush_text(cb) + + def _finish_string(self, screen, cb): + """Dispatch the completed string (OSC/DCS/APC/PM/SOS).""" + st = self.str_type + if st == _OSC: + self._dispatch_osc(screen, cb) + elif st == _DCS: + self._dispatch_dcs(screen, cb) + elif st == _APC: + # Check for graphics command (_G) + payload = bytes(self.str_buf) + if payload.startswith(b'G'): + self.str_buf = bytearray(payload[1:]) + self._dispatch_graphics(screen, cb) + else: + code = payload[0] if payload else 0 + self._emit(cb, 'Unrecognized APC code: 0x%02x' % code) + elif st == _PM: + code = self.str_buf[0] if self.str_buf else 0 + self._emit(cb, 'Unrecognized PM code: 0x%02x' % code) + elif st == _SOS: + code = self.str_buf[0] if self.str_buf else 0 + self._emit(cb, 'Unrecognized SOS code: 0x%02x' % code) + self.str_buf.clear() + + +# Module-level parse_bytes function +def parse_bytes(screen, data, callback=None): + """Parse *data* and emit events to *callback*. + + Maintains parser state on the screen object so partial sequences + split across ``parse_bytes`` invocations are handled correctly. + Also updates screen.callbacks for OSC events (titlebuf, etc.). + """ + if callback is None: + callback = lambda *args: None + if not hasattr(screen, '_kitty_parser'): + screen._kitty_parser = KittyParser() + + def wrapped_cb(wid, *args): + # Update screen callbacks for OSC/DCS events + if args and args[0] == 'set_title': + screen.callbacks.titlebuf.append(args[1]) + elif args and args[0] == 'set_icon': + screen.callbacks.iconbuf += args[1] if isinstance(args[1], str) else str(args[1], 'utf-8') + elif args and args[0] == 'desktop_notify': + screen.callbacks.notifications.append((args[1], args[2])) + elif args and args[0] == 'set_dynamic_color': + val = args[2] if isinstance(args[2], str) else str(args[2], 'utf-8') + screen.callbacks.colorbuf += val + elif args and args[0] == 'handle_remote_print': + try: + from fast_data_types import base64_decode + screen.callbacks.printbuf.append(base64_decode(args[1]).decode('utf-8')) + except Exception: + screen.callbacks.printbuf.append(args[1]) + # Forward to original callback + callback(wid, *args) + + screen._kitty_parser.parse(screen, data, wrapped_cb) diff --git a/tests/extern/kitty/parser.py b/tests/extern/kitty/parser.py new file mode 100644 index 0000000..537e1c0 --- /dev/null +++ b/tests/extern/kitty/parser.py @@ -0,0 +1,995 @@ +#!/usr/bin/env python +# License: GPL v3 Copyright: 2016, Kovid Goyal <kovid at kovidgoyal.net> + +from binascii import hexlify +from functools import partial + +from fast_data_types import ( + CURSOR_BLOCK, + VT_PARSER_BUFFER_SIZE, + base64_decode, + base64_encode, + has_avx2, + has_sse4_2, + test_find_either_of_two_bytes, + test_utf8_decode_to_sentinel, +) + +from kitty_harness import BaseTest, parse_bytes + + +def cnv(x): + if isinstance(x, memoryview): + x = str(x, 'utf-8') + return x + + +class CmdDump(list): + + def __call__(self, window_id, *a): + if a and a[0] == 'bytes': + return + if a and a[0] == 'error': + a = a[1:] + self.append(tuple(map(cnv, a))) + + def get_result(self): + current = '' + q = [] + for args in self: + if args[0] == 'draw': + current += args[1] + else: + if current: + q.append(('draw', current)) + current = '' + q.append(args) + if current: + q.append(('draw', current)) + return tuple(q) + + +class TestParser(BaseTest): + + def create_write_buffer(self, screen): + return screen.test_create_write_buffer() + + def write_bytes(self, screen, write_buf, data): + if isinstance(data, str): + data = data.encode('utf-8') + s = screen.test_commit_write_buffer(data, write_buf) + return data[s:] + + def parse_written_data(self, screen, *cmds): + cd = CmdDump() + screen.test_parse_written_data(cd) + cmds = tuple(('draw', x) if isinstance(x, str) else tuple(map(cnv, x)) for x in cmds) + self.ae(cmds, cd.get_result()) + + def parse_bytes_dump(self, s, x, *cmds): + cd = CmdDump() + if isinstance(x, str): + x = x.encode('utf-8') + cmds = tuple(('draw', x) if isinstance(x, str) else tuple(map(cnv, x)) for x in cmds) + parse_bytes(s, x, cd) + self.ae(cmds, cd.get_result()) + + def test_charsets(self): + s = self.create_screen() + pb = partial(self.parse_bytes_dump, s) + pb(b'\xc3') + pb(b'\xa1', ('draw', b'\xc3\xa1'.decode('utf-8'))) + s = self.create_screen() + pb = partial(self.parse_bytes_dump, s) + pb('\033)0\x0e/_', ('screen_designate_charset', 1, ord('0')), ('screen_change_charset', 1), '/_') + self.ae(str(s.line(0)), '/\xa0') + s = self.create_screen() + pb = partial(self.parse_bytes_dump, s) + pb('\033(0/_', ('screen_designate_charset', 0, ord('0')), '/_') + self.ae(str(s.line(0)), '/\xa0') + + def test_parser_threading(self): + s = self.create_screen() + + self.assertFalse(self.write_bytes(s, self.create_write_buffer(s), 'a\x1b]2;some title')) + b = self.create_write_buffer(s) + self.parse_written_data(s, 'a') + self.assertFalse(self.write_bytes(s, b, ' full\x1b\\')) + self.parse_written_data(s, ('set_title', 'some title full')) + + self.assertFalse(self.write_bytes(s, self.create_write_buffer(s), 'a\x1b]')) + b = self.create_write_buffer(s) + self.parse_written_data(s, 'a') + self.assertFalse(self.write_bytes(s, b, '2;title\x1b\\')) + self.parse_written_data(s, ('set_title', 'title')) + + self.assertFalse(self.write_bytes(s, self.create_write_buffer(s), 'a\x1b')) + b = self.create_write_buffer(s) + self.parse_written_data(s, 'a') + self.assertFalse(self.write_bytes(s, b, ']2;title\x1b\\')) + self.parse_written_data(s, ('set_title', 'title')) + + self.assertFalse(self.write_bytes(s, self.create_write_buffer(s), 'a\x1b]2;some title\x1b')) + b = self.create_write_buffer(s) + self.parse_written_data(s, 'a') + self.assertFalse(self.write_bytes(s, b, '\\b')) + self.parse_written_data(s, ('set_title', 'some title'), 'b') + + self.assertFalse(self.write_bytes(s, self.create_write_buffer(s), '1\x1b')) + self.parse_written_data(s, '1') + self.assertFalse(self.write_bytes(s, self.create_write_buffer(s), 'E2')) + self.parse_written_data(s, ('screen_nel',), ('draw', '2')) + + self.assertFalse(self.write_bytes(s, self.create_write_buffer(s), '1\x1b[2')) + self.parse_written_data(s, '1') + self.assertFalse(self.write_bytes(s, self.create_write_buffer(s), '3mx')) + self.parse_written_data(s, ('select_graphic_rendition', '23'), 'x') + + self.assertFalse(self.write_bytes(s, self.create_write_buffer(s), '1\x1b')) + self.parse_written_data(s, '1') + self.assertFalse(self.write_bytes(s, self.create_write_buffer(s), '[23mx')) + self.parse_written_data(s, ('select_graphic_rendition', '23'), 'x') + + self.assertFalse(self.write_bytes(s, self.create_write_buffer(s), '1\x1b[')) + self.parse_written_data(s, '1') + self.assertFalse(self.write_bytes(s, self.create_write_buffer(s), '23mx')) + self.parse_written_data(s, ('select_graphic_rendition', '23'), 'x') + + # test full write + sz = VT_PARSER_BUFFER_SIZE // 3 + 7 + self.assertFalse(self.write_bytes(s, self.create_write_buffer(s), b'a' * sz)) + self.assertFalse(self.write_bytes(s, self.create_write_buffer(s), b'b' * sz)) + left = self.write_bytes(s, self.create_write_buffer(s), b'c' * sz) + self.assertTrue(len(left), 3 * sz - VT_PARSER_BUFFER_SIZE) + self.assertFalse(self.create_write_buffer(s)) + s.test_parse_written_data() + b = self.create_write_buffer(s) + self.assertTrue(b) + self.write_bytes(s, b, b'') + + def test_base64(self): + for src, expected in { + 'bGlnaHQgdw==': 'light w', + 'bGlnaHQgd28=': 'light wo', + 'bGlnaHQgd29y': 'light wor', + }.items(): + self.ae(base64_decode(src.encode()), expected.encode(), f'Decoding of {src} failed') + self.ae(base64_decode(src.replace('=', '').encode()), expected.encode(), f'Decoding of {src} failed') + self.ae(base64_encode(expected.encode()), src.replace('=', '').encode(), f'Encoding of {expected} failed') + + def test_simple_parsing(self): + s = self.create_screen() + pb = partial(self.parse_bytes_dump, s) + + pb('12', '12') + self.ae(str(s.line(0)), '12') + self.ae(s.cursor.x, 2) + pb('3456', '3456') + self.ae(str(s.line(0)), '12345') + self.ae(str(s.line(1)), '6') + pb(b'\n123\n\r45', ('screen_linefeed',), '123', ('screen_linefeed',), ('screen_carriage_return',), '45') + self.ae(str(s.line(1)), '6') + self.ae(str(s.line(2)), ' 123') + self.ae(str(s.line(3)), '45') + pb(b'\rabcde', ('screen_carriage_return',), 'abcde') + self.ae(str(s.line(3)), 'abcde') + pb('\rßxyz1', ('screen_carriage_return',), 'ßxyz1') + self.ae(str(s.line(3)), 'ßxyz1') + pb('ニチ ', 'ニチ ') + self.ae(str(s.line(4)), 'ニチ ') + s.reset() + self.assertFalse(str(s.line(1)) + str(s.line(2)) + str(s.line(3))) + c1_controls = '\x84\x85\x88\x8d\x8e\x8f\x90\x96\x97\x98\x9a\x9b\x9c\x9d\x9e\x9f' + pb(c1_controls, c1_controls) + self.assertFalse(str(s.line(1)) + str(s.line(2)) + str(s.line(3))) + pb('😀'.encode()[:-1]) + pb('\x1b\x1b%a', '\ufffd', ('Unknown char after ESC: 0x1b',), ('draw', '%a')) + + def test_utf8_simd_decode(self): + def unsupported(which): + return (which == 2 and not has_sse4_2) or (which == 3 and not has_avx2) + + def reset_state(): + test_utf8_decode_to_sentinel(b'', -1) + + def asbytes(x): + if isinstance(x, str): + x = x.encode() + return x + + def t(*a, which=2): + if unsupported(which): + return + + def parse_parts(which): + total_consumed = 0 + esc_found = False + parts = [] + for x in a: + found_sentinel, x, num_consumed = test_utf8_decode_to_sentinel(asbytes(x), which) + total_consumed += num_consumed + if found_sentinel: + esc_found = found_sentinel + parts.append(x) + return esc_found, ''.join(parts), total_consumed + + reset_state() + expected = parse_parts(1) + reset_state() + actual = parse_parts(which) + self.ae(expected, actual, msg=f'Failed for {a} with {which=}\n{expected!r} !=\n{actual!r}') + return actual + + def double_test(x): + for which in (2, 3): + t(x, which=which) + t(x*2, which=3) + reset_state() + + # incomplete trailer at end of vector + t("a"*10 + "😸😸" + "b"*15) + + x = double_test + x('2:α3') + x('2:α\x1b3') + x('2:α3:≤4:😸|') + x('abcd1234efgh5678') + x('abc\x1bd1234efgh5678') + x('abcd1234efgh5678ijklABCDmnopEFGH') + + for which in (2, 3): + x = partial(t, which=which) + x('abcdef', 'ghijk') + x('2:α3', ':≤4:😸|') + # trailing incomplete sequence + for prefix in (b'abcd', '😸'.encode()): + for suffix in (b'1234', '😸'.encode()): + x(prefix + b'\xf0\x9f', b'\x98\xb8' + suffix) + x(prefix + b'\xf0\x9f\x9b', b'\xb8' + suffix) + x(prefix + b'\xf0', b'\x9f\x98\xb8' + suffix) + x(prefix + b'\xc3', b'\xa4' + suffix) + x(prefix + b'\xe2', b'\x89\xa4' + suffix) + x(prefix + b'\xe2\x89', b'\xa4' + suffix) + + def test_expected(src, expected, which=2): + if unsupported(which): + return + reset_state() + _, actual, _ = t(b'filler' + asbytes(src), which=which) + expected = 'filler' + expected + self.ae(expected, actual, f'Failed for: {src!r} with {which=}') + + for which in (1, 2, 3): + pb = partial(test_expected, which=which) + pb('ニチ', 'ニチ') + pb('\x84\x85', '\x84\x85') + pb('\x84\x85', '\x84\x85') + pb('\uf4df', '\uf4df') + pb('\uffff', '\uffff') + pb('\0', '\0') + pb(chr(0x10ffff), chr(0x10ffff)) + # Kitty's UTF-8 decoding uses `U+FFFD substitution of maximal subparts + # <https://www.unicode.org/versions/Unicode16.0.0/core-spec/chapter-3/#G66453>`_, + # same as in the WHATWG Encoding Standard. + # This means that ill-formed sequences may be replaced by multiple + # U+FFFD REPLACEMENT CHARACTERs. + pb(b'abcd\xf51234', 'abcd\ufffd1234') # bytes > 0xf4 + pb(b'abcd\xff1234', 'abcd\ufffd1234') # bytes > 0xf4 + pb(b'"\xbf"', '"\ufffd"') + pb(b'"\x80"', '"\ufffd"') + pb(b'"\x80\xbf"', '"\ufffd\ufffd"') + pb(b'"\x80\xbf\x80"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xc0 "', '"\ufffd "') + pb(b'"\xfe"', '"\ufffd"') + pb(b'"\xff"', '"\ufffd"') + pb(b'"\xff\xfe"', '"\ufffd\ufffd"') + pb(b'"\xfe\xfe\xff\xff"', '"\ufffd\ufffd\ufffd\ufffd"') + pb(b'"\xef\xbf"', '"\ufffd"') + pb(b'"\xe0\xa0"', '"\ufffd"') + pb(b'"\xf0\x9f\x98"', '"\ufffd"') + pb(b'"\xef\x93\x94\x95"', '"\uf4d4\ufffd"') + + # Lone continuation bytes with no leading starts + pb(b'"\xbf"', '"\ufffd"') + pb(b'"\x80"', '"\ufffd"') + + # Multiple lone continuation bytes + pb(b'"\x80\xbf"', '"\ufffd\ufffd"') + pb(b'"\x80\xbf\x80"', '"\ufffd\ufffd\ufffd"') + + # Lone starter byte of 2-byte sequence + pb(b'"\xc0 "', '"\ufffd "') + + # Single never-valid bytes + pb(b'"\xfe"', '"\ufffd"') + pb(b'"\xff"', '"\ufffd"') + + # Multiple never-valid bytes + pb(b'"\xff\xfe"', '"\ufffd\ufffd"') + pb(b'"\xfe\xfe\xff\xff"', '"\ufffd\ufffd\ufffd\ufffd"') + + # Truncated 2-byte sequence (only 1 byte) + pb(b'"\xc2"', '"\ufffd"') + + # Truncated 3-byte sequences (only 2 bytes) + pb(b'"\xef\xbf"', '"\ufffd"') + pb(b'"\xe0\xa0"', '"\ufffd"') + + # Truncated 4-byte sequence (only 2 or 3 bytes) + pb(b'"\xf0\x9f"', '"\ufffd"') + pb(b'"\xf0\x9f\x98"', '"\ufffd"') + + # Bad continuation byte (restored as ASCII) + pb(b'"\xe1\x28\xa1"', '"\ufffd(\ufffd"') # ) + + # Overlong 2-byte sequence for U+0000 (should be `0x00`) + pb(b'"\xc0\x80"', '"\ufffd\ufffd"') + + # Overlong 3-byte sequence for U+0000 (violates boundary) + pb(b'"\xe0\x80\x80"', '"\ufffd\ufffd\ufffd"') + + # Overlong 4-byte sequence for U+0000 (violates boundary) + pb(b'"\xf0\x80\x80\x80"', '"\ufffd\ufffd\ufffd\ufffd"') + + # High surrogate code point + pb(b'"\xed\xa0\x80"', '"\ufffd\ufffd\ufffd"') + + # Low surrogate code point + pb(b'"\xed\xb0\x80"', '"\ufffd\ufffd\ufffd"') + + # Too large starter byte + pb(b'"\xff\x80\x80\x80"', '"\ufffd\ufffd\ufffd\ufffd"') + + # The following boundary cases come from the table of well-formed UTF-8 byte sequences + # <https://www.unicode.org/versions/Unicode16.0.0/core-spec/chapter-3/#G27506>`_. + # For continuation bytes, both 0xC0 and 0xC2 are tested as values that exceed the valid maximum. + # This is because 0xC0 is an invalid starter byte, but 0xC2 is also a starter byte for 2-byte sequences. + # simd-string-impl.h prefers classifying bytes as starter bytes when possible (e.g., in "\xf0\x90\xc2\x80"). + # The tests need to check that simd-string-impl.h correctly detects + # starter bytes that are actually invalid continution bytes, like 0xC2. + + # Boundary cases: 2-byte sequences + pb(b'"\xc1\x7f"', '"\ufffd\x7f"') + pb(b'"\xc1\x80"', '"\ufffd\ufffd"') + pb(b'"\xc1\xbf"', '"\ufffd\ufffd"') + pb(b'"\xc1\xc0"', '"\ufffd\ufffd"') + pb(b'"\xc1\xc2"', '"\ufffd\ufffd"') + pb(b'"\xc2\x7f"', '"\ufffd\x7f"') + pb(b'"\xc2\x80"', '"\x80"') + pb(b'"\xc2\xbf"', '"\xbf"') + pb(b'"\xc2\xc0"', '"\ufffd\ufffd"') + pb(b'"\xc2\xc2"', '"\ufffd\ufffd"') + pb(b'"\xdf\x7f"', '"\ufffd\x7f"') + pb(b'"\xdf\x80"', '"\u07c0"') + pb(b'"\xdf\xbf"', '"\u07ff"') + pb(b'"\xdf\xc0"', '"\ufffd\ufffd"') + pb(b'"\xdf\xc2"', '"\ufffd\ufffd"') + + # Boundary cases: 3-byte sequences starting with 0xE0 + pb(b'"\xe0\x9f\x80"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xe0\xa0\x7f"', '"\ufffd\x7f"') + pb(b'"\xe0\xa0\x80"', '"\u0800"') + pb(b'"\xe0\xa0\xbf"', '"\u083f"') + pb(b'"\xe0\xa0\xc0"', '"\ufffd\ufffd"') + pb(b'"\xe0\xa0\xc2"', '"\ufffd\ufffd"') + pb(b'"\xe0\xbf\x7f"', '"\ufffd\x7f"') + pb(b'"\xe0\xbf\x80"', '"\u0fc0"') + pb(b'"\xe0\xbf\xbf"', '"\u0fff"') + pb(b'"\xe0\xbf\xc0"', '"\ufffd\ufffd"') + pb(b'"\xe0\xbf\xc2"', '"\ufffd\ufffd"') + pb(b'"\xe0\xc0\x80"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xe0\xc2\x80"', '"\ufffd\x80"') + + # Boundary cases: 3-byte sequences starting with 0xE1..0xEC + pb(b'"\xe1\x7f\x80"', '"\ufffd\x7f\ufffd"') + pb(b'"\xe1\x80\x7f"', '"\ufffd\x7f"') + pb(b'"\xe1\x80\x80"', '"\u1000"') + pb(b'"\xe1\x80\xbf"', '"\u103f"') + pb(b'"\xe1\x80\xc0"', '"\ufffd\ufffd"') + pb(b'"\xe1\x80\xc2"', '"\ufffd\ufffd"') + pb(b'"\xe1\xbf\x7f"', '"\ufffd\x7f"') + pb(b'"\xe1\xbf\x80"', '"\u1fc0"') + pb(b'"\xe1\xbf\xbf"', '"\u1fff"') + pb(b'"\xe1\xbf\xc0"', '"\ufffd\ufffd"') + pb(b'"\xe1\xbf\xc2"', '"\ufffd\ufffd"') + pb(b'"\xe1\xc0\x80"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xe1\xc2\x80"', '"\ufffd\x80"') + pb(b'"\xec\x7f\x80"', '"\ufffd\x7f\ufffd"') + pb(b'"\xec\x80\x7f"', '"\ufffd\x7f"') + pb(b'"\xec\x80\x80"', '"\uc000"') + pb(b'"\xec\x80\xbf"', '"\uc03f"') + pb(b'"\xec\x80\xc0"', '"\ufffd\ufffd"') + pb(b'"\xec\x80\xc2"', '"\ufffd\ufffd"') + pb(b'"\xec\xbf\x7f"', '"\ufffd\x7f"') + pb(b'"\xec\xbf\x80"', '"\ucfc0"') + pb(b'"\xec\xbf\xbf"', '"\ucfff"') + pb(b'"\xec\xbf\xc0"', '"\ufffd\ufffd"') + pb(b'"\xec\xbf\xc2"', '"\ufffd\ufffd"') + pb(b'"\xec\xc0\x80"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xec\xc2\x80"', '"\ufffd\x80"') + + # Boundary cases: 3-byte sequences starting with 0xED + pb(b'"\xed\x7f\x80"', '"\ufffd\x7f\ufffd"') + pb(b'"\xed\x80\x7f"', '"\ufffd\x7f"') + pb(b'"\xed\x80\x80"', '"\ud000"') + pb(b'"\xed\x80\xbf"', '"\ud03f"') + pb(b'"\xed\x80\xc0"', '"\ufffd\ufffd"') + pb(b'"\xed\x80\xc2"', '"\ufffd\ufffd"') + pb(b'"\xed\x9f\x7f"', '"\ufffd\x7f"') + pb(b'"\xed\x9f\x80"', '"\ud7c0"') + pb(b'"\xed\x9f\xbf"', '"\ud7ff"') + pb(b'"\xed\x9f\xc0"', '"\ufffd\ufffd"') + pb(b'"\xed\x9f\xc2"', '"\ufffd\ufffd"') + + # Boundary cases: 3-byte sequences starting with 0xEE..0xEF + pb(b'"\xed\xa0\x80"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xee\x7f\x80"', '"\ufffd\x7f\ufffd"') + pb(b'"\xee\x80\x7f"', '"\ufffd\x7f"') + pb(b'"\xee\x80\x80"', '"\ue000"') + pb(b'"\xee\x80\xbf"', '"\ue03f"') + pb(b'"\xee\x80\xc0"', '"\ufffd\ufffd"') + pb(b'"\xee\x80\xc2"', '"\ufffd\ufffd"') + pb(b'"\xee\xbf\x7f"', '"\ufffd\x7f"') + pb(b'"\xee\xbf\x80"', '"\uefc0"') + pb(b'"\xee\xbf\xbf"', '"\uefff"') + pb(b'"\xee\xbf\xc0"', '"\ufffd\ufffd"') + pb(b'"\xee\xbf\xc2"', '"\ufffd\ufffd"') + pb(b'"\xee\xc0\x80"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xee\xc2\x80"', '"\ufffd\x80"') + pb(b'"\xef\x7f\x80"', '"\ufffd\x7f\ufffd"') + pb(b'"\xef\x80\x7f"', '"\ufffd\x7f"') + pb(b'"\xef\x80\x80"', '"\uf000"') + pb(b'"\xef\x80\xbf"', '"\uf03f"') + pb(b'"\xef\x80\xc0"', '"\ufffd\ufffd"') + pb(b'"\xef\x80\xc2"', '"\ufffd\ufffd"') + pb(b'"\xef\xbf\x7f"', '"\ufffd\x7f"') + pb(b'"\xef\xbf\x80"', '"\uffc0"') + pb(b'"\xef\xbf\xbf"', '"\uffff"') + pb(b'"\xef\xbf\xc0"', '"\ufffd\ufffd"') + pb(b'"\xef\xbf\xc2"', '"\ufffd\ufffd"') + pb(b'"\xef\xc0\x80"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xef\xc2\x80"', '"\ufffd\x80"') + + # Boundary cases: 4-byte sequences starting with 0xF0 + pb(b'"\xf0\x8f\x80\x80"', '"\ufffd\ufffd\ufffd\ufffd"') + pb(b'"\xf0\x90\x7f\x80"', '"\ufffd\x7f\ufffd"') + pb(b'"\xf0\x90\x80\x7f"', '"\ufffd\x7f"') + pb(b'"\xf0\x90\x80\x80"', '"\U00010000"') + pb(b'"\xf0\x90\x80\xbf"', '"\U0001003f"') + pb(b'"\xf0\x90\x80\xc0"', '"\ufffd\ufffd"') + pb(b'"\xf0\x90\x80\xc2"', '"\ufffd\ufffd"') + pb(b'"\xf0\x90\xbf\x7f"', '"\ufffd\x7f"') + pb(b'"\xf0\x90\xbf\x80"', '"\U00010fc0"') + pb(b'"\xf0\x90\xbf\xbf"', '"\U00010fff"') + pb(b'"\xf0\x90\xbf\xc0"', '"\ufffd\ufffd"') + pb(b'"\xf0\x90\xbf\xc2"', '"\ufffd\ufffd"') + pb(b'"\xf0\x90\xc0\x7f"', '"\ufffd\ufffd\x7f"') + pb(b'"\xf0\x90\xc0\x80"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf0\x90\xc0\xbf"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf0\x90\xc0\xc0"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf0\x90\xc0\xc2"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf0\x90\xc2\x7f"', '"\ufffd\ufffd\x7f"') + pb(b'"\xf0\x90\xc2\x80"', '"\ufffd\x80"') + pb(b'"\xf0\x90\xc2\xbf"', '"\ufffd\xbf"') + pb(b'"\xf0\x90\xc2\xc0"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf0\x90\xc2\xc2"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf0\xbf\x7f\x80"', '"\ufffd\x7f\ufffd"') + pb(b'"\xf0\xbf\x80\x7f"', '"\ufffd\x7f"') + pb(b'"\xf0\xbf\x80\x80"', '"\U0003f000"') + pb(b'"\xf0\xbf\x80\xbf"', '"\U0003f03f"') + pb(b'"\xf0\xbf\x80\xc0"', '"\ufffd\ufffd"') + pb(b'"\xf0\xbf\x80\xc2"', '"\ufffd\ufffd"') + pb(b'"\xf0\xbf\xbf\x7f"', '"\ufffd\x7f"') + pb(b'"\xf0\xbf\xbf\x80"', '"\U0003ffc0"') + pb(b'"\xf0\xbf\xbf\xbf"', '"\U0003ffff"') + pb(b'"\xf0\xbf\xbf\xc0"', '"\ufffd\ufffd"') + pb(b'"\xf0\xbf\xbf\xc2"', '"\ufffd\ufffd"') + pb(b'"\xf0\xbf\xc0\x7f"', '"\ufffd\ufffd\x7f"') + pb(b'"\xf0\xbf\xc0\x80"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf0\xbf\xc0\xbf"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf0\xbf\xc0\xc0"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf0\xbf\xc0\xc2"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf0\xbf\xc2\x7f"', '"\ufffd\ufffd\x7f"') + pb(b'"\xf0\xbf\xc2\x80"', '"\ufffd\x80"') + pb(b'"\xf0\xbf\xc2\xbf"', '"\ufffd\xbf"') + pb(b'"\xf0\xbf\xc2\xc0"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf0\xbf\xc2\xc2"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf0\xc0\x80\x80"', '"\ufffd\ufffd\ufffd\ufffd"') + + # Boundary cases: 4-byte sequences starting with 0xF1..0xF3 + pb(b'"\xf1\x7f\x80\x80"', '"\ufffd\x7f\ufffd\ufffd"') + pb(b'"\xf1\x80\x7f\x80"', '"\ufffd\x7f\ufffd"') + pb(b'"\xf1\x80\x80\x7f"', '"\ufffd\x7f"') + pb(b'"\xf1\x80\x80\x80"', '"\U00040000"') + pb(b'"\xf1\x80\x80\xbf"', '"\U0004003f"') + pb(b'"\xf1\x80\x80\xc0"', '"\ufffd\ufffd"') + pb(b'"\xf1\x80\x80\xc2"', '"\ufffd\ufffd"') + pb(b'"\xf1\x80\xbf\x7f"', '"\ufffd\x7f"') + pb(b'"\xf1\x80\xbf\x80"', '"\U00040fc0"') + pb(b'"\xf1\x80\xbf\xbf"', '"\U00040fff"') + pb(b'"\xf1\x80\xbf\xc0"', '"\ufffd\ufffd"') + pb(b'"\xf1\x80\xbf\xc2"', '"\ufffd\ufffd"') + pb(b'"\xf1\x80\xc0\x7f"', '"\ufffd\ufffd\x7f"') + pb(b'"\xf1\x80\xc0\x80"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf1\x80\xc0\xbf"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf1\x80\xc0\xc0"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf1\x80\xc2\x7f"', '"\ufffd\ufffd\x7f"') + pb(b'"\xf1\x80\xc2\x80"', '"\ufffd\x80"') + pb(b'"\xf1\x80\xc2\xbf"', '"\ufffd\xbf"') + pb(b'"\xf1\x80\xc2\xc0"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf1\xbf\x7f\x80"', '"\ufffd\x7f\ufffd"') + pb(b'"\xf1\xbf\x80\x7f"', '"\ufffd\x7f"') + pb(b'"\xf1\xbf\x80\x80"', '"\U0007f000"') + pb(b'"\xf1\xbf\x80\xbf"', '"\U0007f03f"') + pb(b'"\xf1\xbf\x80\xc0"', '"\ufffd\ufffd"') + pb(b'"\xf1\xbf\x80\xc2"', '"\ufffd\ufffd"') + pb(b'"\xf1\xbf\xbf\x7f"', '"\ufffd\x7f"') + pb(b'"\xf1\xbf\xbf\x80"', '"\U0007ffc0"') + pb(b'"\xf1\xbf\xbf\xbf"', '"\U0007ffff"') + pb(b'"\xf1\xbf\xbf\xc0"', '"\ufffd\ufffd"') + pb(b'"\xf1\xbf\xbf\xc2"', '"\ufffd\ufffd"') + pb(b'"\xf1\xbf\xc0\x7f"', '"\ufffd\ufffd\x7f"') + pb(b'"\xf1\xbf\xc0\x80"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf1\xbf\xc0\xbf"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf1\xbf\xc0\xc0"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf1\xbf\xc0\xc2"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf1\xbf\xc2\x7f"', '"\ufffd\ufffd\x7f"') + pb(b'"\xf1\xbf\xc2\x80"', '"\ufffd\x80"') + pb(b'"\xf1\xbf\xc2\xbf"', '"\ufffd\xbf"') + pb(b'"\xf1\xbf\xc2\xc0"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf1\xbf\xc2\xc2"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf1\xc0\x80\x80"', '"\ufffd\ufffd\ufffd\ufffd"') + pb(b'"\xf1\xc2\x80\x80"', '"\ufffd\x80\ufffd"') + pb(b'"\xf3\x7f\x80\x80"', '"\ufffd\x7f\ufffd\ufffd"') + pb(b'"\xf3\x80\x7f\x80"', '"\ufffd\x7f\ufffd"') + pb(b'"\xf3\x80\x80\x7f"', '"\ufffd\x7f"') + pb(b'"\xf3\x80\x80\x80"', '"\U000c0000"') + pb(b'"\xf3\x80\x80\xbf"', '"\U000c003f"') + pb(b'"\xf3\x80\x80\xc0"', '"\ufffd\ufffd"') + pb(b'"\xf3\x80\x80\xc2"', '"\ufffd\ufffd"') + pb(b'"\xf3\x80\xbf\x7f"', '"\ufffd\x7f"') + pb(b'"\xf3\x80\xbf\x80"', '"\U000c0fc0"') + pb(b'"\xf3\x80\xbf\xbf"', '"\U000c0fff"') + pb(b'"\xf3\x80\xbf\xc0"', '"\ufffd\ufffd"') + pb(b'"\xf3\x80\xbf\xc2"', '"\ufffd\ufffd"') + pb(b'"\xf3\x80\xc0\x7f"', '"\ufffd\ufffd\x7f"') + pb(b'"\xf3\x80\xc0\x80"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf3\x80\xc0\xbf"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf3\x80\xc0\xc0"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf3\x80\xc0\xc2"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf3\x80\xc2\x7f"', '"\ufffd\ufffd\x7f"') + pb(b'"\xf3\x80\xc2\x80"', '"\ufffd\x80"') + pb(b'"\xf3\x80\xc2\xbf"', '"\ufffd\xbf"') + pb(b'"\xf3\x80\xc2\xc0"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf3\x80\xc2\xc2"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf3\xbf\x7f\x80"', '"\ufffd\x7f\ufffd"') + pb(b'"\xf3\xbf\x80\x7f"', '"\ufffd\x7f"') + pb(b'"\xf3\xbf\x80\x80"', '"\U000ff000"') + pb(b'"\xf3\xbf\x80\xbf"', '"\U000ff03f"') + pb(b'"\xf3\xbf\x80\xc0"', '"\ufffd\ufffd"') + pb(b'"\xf3\xbf\x80\xc2"', '"\ufffd\ufffd"') + pb(b'"\xf3\xbf\xbf\x7f"', '"\ufffd\x7f"') + pb(b'"\xf3\xbf\xbf\x80"', '"\U000fffc0"') + pb(b'"\xf3\xbf\xbf\xbf"', '"\U000fffff"') + pb(b'"\xf3\xbf\xbf\xc0"', '"\ufffd\ufffd"') + pb(b'"\xf3\xbf\xbf\xc2"', '"\ufffd\ufffd"') + pb(b'"\xf3\xbf\xc0\x7f"', '"\ufffd\ufffd\x7f"') + pb(b'"\xf3\xbf\xc0\x80"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf3\xbf\xc0\xbf"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf3\xbf\xc0\xc0"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf3\xbf\xc0\xc2"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf3\xbf\xc2\x7f"', '"\ufffd\ufffd\x7f"') + pb(b'"\xf3\xbf\xc2\x80"', '"\ufffd\x80"') + pb(b'"\xf3\xbf\xc2\xbf"', '"\ufffd\xbf"') + pb(b'"\xf3\xbf\xc2\xc0"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf3\xbf\xc2\xc2"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf3\xc0\x80\x80"', '"\ufffd\ufffd\ufffd\ufffd"') + pb(b'"\xf3\xc2\x80\x80"', '"\ufffd\x80\ufffd"') + + # Boundary cases: 4-byte sequences starting with 0xF4 + pb(b'"\xf4\x7f\x80\x80"', '"\ufffd\x7f\ufffd\ufffd"') + pb(b'"\xf4\x80\x7f\x80"', '"\ufffd\x7f\ufffd"') + pb(b'"\xf4\x80\x80\x7f"', '"\ufffd\x7f"') + pb(b'"\xf4\x80\x80\x80"', '"\U00100000"') + pb(b'"\xf4\x80\x80\xbf"', '"\U0010003f"') + pb(b'"\xf4\x80\x80\xc0"', '"\ufffd\ufffd"') + pb(b'"\xf4\x80\x80\xc2"', '"\ufffd\ufffd"') + pb(b'"\xf4\x80\xbf\x7f"', '"\ufffd\x7f"') + pb(b'"\xf4\x80\xbf\x80"', '"\U00100fc0"') + pb(b'"\xf4\x80\xbf\xbf"', '"\U00100fff"') + pb(b'"\xf4\x80\xbf\xc0"', '"\ufffd\ufffd"') + pb(b'"\xf4\x80\xbf\xc2"', '"\ufffd\ufffd"') + pb(b'"\xf4\x80\xc0\x7f"', '"\ufffd\ufffd\x7f"') + pb(b'"\xf4\x80\xc0\x80"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf4\x80\xc0\xbf"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf4\x80\xc0\xc0"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf4\x80\xc0\xc2"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf4\x80\xc2\x7f"', '"\ufffd\ufffd\x7f"') + pb(b'"\xf4\x80\xc2\x80"', '"\ufffd\x80"') + pb(b'"\xf4\x80\xc2\xbf"', '"\ufffd\xbf"') + pb(b'"\xf4\x80\xc2\xc0"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf4\x80\xc2\xc2"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf4\x8f\x7f\x80"', '"\ufffd\x7f\ufffd"') + pb(b'"\xf4\x8f\x80\x7f"', '"\ufffd\x7f"') + pb(b'"\xf4\x8f\x80\x80"', '"\U0010f000"') + pb(b'"\xf4\x8f\x80\xbf"', '"\U0010f03f"') + pb(b'"\xf4\x8f\x80\xc0"', '"\ufffd\ufffd"') + pb(b'"\xf4\x8f\x80\xc2"', '"\ufffd\ufffd"') + pb(b'"\xf4\x8f\xbf\x7f"', '"\ufffd\x7f"') + pb(b'"\xf4\x8f\xbf\x80"', '"\U0010ffc0"') + pb(b'"\xf4\x8f\xbf\xbf"', '"\U0010ffff"') + pb(b'"\xf4\x8f\xbf\xc0"', '"\ufffd\ufffd"') + pb(b'"\xf4\x8f\xbf\xc2"', '"\ufffd\ufffd"') + pb(b'"\xf4\x8f\xc0\x7f"', '"\ufffd\ufffd\x7f"') + pb(b'"\xf4\x8f\xc0\x80"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf4\x8f\xc0\xbf"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf4\x8f\xc0\xc0"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf4\x8f\xc0\xc2"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf4\x8f\xc2\x7f"', '"\ufffd\ufffd\x7f"') + pb(b'"\xf4\x8f\xc2\x80"', '"\ufffd\x80"') + pb(b'"\xf4\x8f\xc2\xbf"', '"\ufffd\xbf"') + pb(b'"\xf4\x8f\xc2\xc0"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf4\x8f\xc2\xc2"', '"\ufffd\ufffd\ufffd"') + pb(b'"\xf4\x90\x80\x80"', '"\ufffd\ufffd\ufffd\ufffd"') + pb(b'"\xf5\x80\x80\x80"', '"\ufffd\ufffd\ufffd\ufffd"') + + # Boundary case: too large codepoint (> U+10FFFF) + pb(b'"\xf5\x80\x80\x80"', '"\ufffd\ufffd\ufffd\ufffd"') + + + def test_find_either_of_two_bytes(self): + sizes = [] + if has_sse4_2: + sizes.append(2) + if has_avx2: + sizes.append(3) + sizes.append(0) + + def test(buf, a, b, align_offset=0): + a_, b_ = ord(a), ord(b) + expected = test_find_either_of_two_bytes(buf, a_, b_, 1, 0) + for sz in sizes: + actual = test_find_either_of_two_bytes(buf, a_, b_, sz, align_offset) + self.ae(expected, actual, f'Failed for: {buf!r} {a=} {b=} at {sz=} and {align_offset=}') + + q = 'abc' + for off in range(32): + test(q, '<', '>', off) + test(q, ' ', 'b', off) + test(q, '<', 'a', off) + test(q, '<', 'b', off) + test(q, 'c', '>', off) + + def tests(buf, a, b): + for sz in (0, 16, 32, 64, 79): + buf = (' ' * sz) + buf + for align_offset in range(32): + test(buf, a, b, align_offset) + tests("", '<', '>') + tests("a", '\0', '\0') + tests("a", '<', '>') + tests("dsdfsfa", '1', 'a') + tests("xa", 'a', 'a') + tests("bbb", 'a', '1') + tests("bba", 'a', '<') + tests("baa", '>', 'a') + + def test_esc_codes(self): + s = self.create_screen() + pb = partial(self.parse_bytes_dump, s) + pb('12\033Da', '12', ('screen_index',), 'a') + self.ae(str(s.line(0)), '12') + self.ae(str(s.line(1)), ' a') + pb('\033xa', ('Unknown char after ESC: 0x%x' % ord('x'),), 'a') + pb('\033c123', ('screen_reset', ), '123') + self.ae(str(s.line(0)), '123') + pb('\033.\033a', ('Unhandled charset related escape code: 0x2e 0x1b',), 'a') + + def test_csi_codes(self): + s = self.create_screen() + pb = partial(self.parse_bytes_dump, s) + pb('abcde', 'abcde') + s.cursor_move(5) + pb('x\033[2@y', 'x', ('screen_insert_characters', 2), 'y') + self.ae(str(s.line(0)), 'xy bc') + pb('x\033[2;7@y', 'x', ('CSI code @ has 2 > 1 parameters',), 'y') + pb('x\033[2;-7@y', 'x', ('CSI code @ has 2 > 1 parameters',), 'y') + pb('x\033[-0001234567890@y', 'x', ('CSI code @ is not allowed to have negative parameter (-1234567890)',), 'y') + pb('x\033[2-3@y', 'x', ('Invalid character in CSI: 3 (0x33), ignoring the sequence',), '@y') + pb('x\033[@y', 'x', ('screen_insert_characters', 1), 'y') + pb('x\033[345@y', 'x', ('screen_insert_characters', 345), 'y') + pb('x\033[345;@y', 'x', ('screen_insert_characters', 345), 'y') + pb('\033[H', ('screen_cursor_position', 1, 1)) + self.ae(s.cursor.x, 0), self.ae(s.cursor.y, 0) + pb('\033[4H', ('screen_cursor_position', 4, 1)) + pb('\033[4;0H', ('screen_cursor_position', 4, 0)) + pb('\033[3;2H', ('screen_cursor_position', 3, 2)) + pb('\033[3;2;H', ('screen_cursor_position', 3, 2)) + pb('\033[00000000003;0000000000000002H', ('screen_cursor_position', 3, 2)) + self.ae(s.cursor.x, 1), self.ae(s.cursor.y, 2) + pb('\033[0001234567890H', ('screen_cursor_position', 1234567890, 1)) + pb('\033[J', ('screen_erase_in_display', 0, 0)) + pb('\033[?J', ('screen_erase_in_display', 0, 1)) + pb('\033[?2J', ('screen_erase_in_display', 2, 1)) + pb('\033[h') + pb('\033[20;4h', ('screen_set_mode', 20, 0), ('screen_set_mode', 4, 0)) + pb('\033[?1000;1004h', ('screen_set_mode', 1000, 1), ('screen_set_mode', 1004, 1)) + pb('\033[20;4;20l', ('screen_reset_mode', 20, 0), ('screen_reset_mode', 4, 0), ('screen_reset_mode', 20, 0)) + pb('\033[=c', ('report_device_attributes', 0, 61)) + s.reset() + + def sgr(*params): + return (('select_graphic_rendition', f'{x}') for x in params) + + pb('\033[1;2;3;4;7;9;34;44m', *sgr('1;2;3;4;7;9;34;44')) + for attr in 'bold italic reverse strikethrough dim'.split(): + self.assertTrue(getattr(s.cursor, attr), attr) + self.ae(s.cursor.decoration, 1) + self.ae(s.cursor.fg, 4 << 8 | 1) + self.ae(s.cursor.bg, 4 << 8 | 1) + pb('\033[38;5;1;48;5;7m', ('select_graphic_rendition', '38:5:1'), ('select_graphic_rendition', '48:5:7')) + self.ae(s.cursor.fg, 1 << 8 | 1) + self.ae(s.cursor.bg, 7 << 8 | 1) + pb('\033[38;2;1;2;3;48;2;7;8;9m', ('select_graphic_rendition', '38:2:1:2:3'), ('select_graphic_rendition', '48:2:7:8:9')) + self.ae(s.cursor.fg, 1 << 24 | 2 << 16 | 3 << 8 | 2) + self.ae(s.cursor.bg, 7 << 24 | 8 << 16 | 9 << 8 | 2) + pb('\033[0;2m', *sgr('0;2')) + pb('\033[;2m', *sgr('0;2')) + pb('\033[m', *sgr('0')) + pb('\033[1;;2m', *sgr('1;0;2')) + pb('\033[38;5;1m', ('select_graphic_rendition', '38:5:1')) + pb('\033[58;2;1;2;3m', ('select_graphic_rendition', '58:2:1:2:3')) + pb('\033[38;2;1;2;3m', ('select_graphic_rendition', '38:2:1:2:3')) + pb('\033[1001:2:1:2:3m', ('select_graphic_rendition', '1001:2:1:2:3')) + pb('\033[38:2:1:2:3;48:5:9;58;5;7m', ( + 'select_graphic_rendition', '38:2:1:2:3'), ('select_graphic_rendition', '48:5:9'), ('select_graphic_rendition', '58:5:7')) + s.reset() + pb('\033[1;2;3;4:5;7;9;34;44m', *sgr('1;2;3', '4:5', '7;9;34;44')) + for attr in 'bold italic reverse strikethrough dim'.split(): + self.assertTrue(getattr(s.cursor, attr), attr) + self.ae(s.cursor.decoration, 5) + c = s.callbacks + pb('\033[5n', ('report_device_status', 5, 0)) + self.ae(c.wtcbuf, b'\033[0n') + c.clear() + pb('\033[6n', ('report_device_status', 6, 0)) + self.ae(c.wtcbuf, b'\033[1;1R') + pb('12345', '12345') + c.clear() + pb('\033[6n', ('report_device_status', 6, 0)) + self.ae(c.wtcbuf, b'\033[2;1R') + c.clear() + s.cursor_key_mode = True + pb('\033[?1$p', ('report_mode_status', 1, 1)) + self.ae(c.wtcbuf, b'\033[?1;1$y') + pb('\033[?1l', ('screen_reset_mode', 1, 1)) + self.assertFalse(s.cursor_key_mode) + c.clear() + pb('\033[?1$p', ('report_mode_status', 1, 1)) + self.ae(c.wtcbuf, b'\033[?1;2$y') + pb('\033[2;4r', ('screen_set_margins', 2, 4)) + c.clear() + pb('\033[14t', ('screen_report_size', 14, 0)) + pb('\033[14;2t', ('screen_report_size', 14, 2)) + self.ae(c.wtcbuf, b'\033[4;100;50t\033[4;100;50t') + self.ae(s.margin_top, 1), self.ae(s.margin_bottom, 3) + pb('\033[r', ('screen_set_margins', 0, 0)) + self.ae(s.margin_top, 0), self.ae(s.margin_bottom, 4) + pb('\033[1 q', ('screen_set_cursor', 1, ord(' '))) + self.assertTrue(s.cursor.blink) + self.ae(s.cursor.shape, CURSOR_BLOCK) + + s.reset() + pb('\033[3 @', ('Shift left escape code not implemented',)) + pb('\033[3 A', ('Shift right escape code not implemented',)) + pb('\033[3;4 S', ('Select presentation directions escape code not implemented',)) + pb('\033[1T', ('screen_reverse_scroll', 1)) + pb('\033[T', ('screen_reverse_scroll', 1)) + pb('\033[+T', ('screen_reverse_scroll_and_fill_from_scrollback', 1)) + + c.clear() + pb('\033[?2026$p', ('report_mode_status', 2026, 1)) + self.ae(c.wtcbuf, b'\x1b[?2026;2$y') + c.clear() + pb('\033[?2026h', ('screen_set_mode', 2026, 1)) + pb('\033[?2026$p', ('report_mode_status', 2026, 1)) + self.ae(c.wtcbuf, b'\x1b[?2026;1$y') + pb('\033[?2026l', ('screen_reset_mode', 2026, 1)) + c.clear() + pb('\033[?2026$p', ('report_mode_status', 2026, 1)) + self.ae(c.wtcbuf, b'\x1b[?2026;2$y') + + def test_csi_code_rep(self): + s = self.create_screen(8) + pb = partial(self.parse_bytes_dump, s) + pb('\033[1b', ('screen_repeat_character', 1)) + self.ae(str(s.line(0)), '') + pb('x\033[7b', 'x', ('screen_repeat_character', 7)) + self.ae(str(s.line(0)), 'xxxxxxxx') + pb('\033[1;3H', ('screen_cursor_position', 1, 3)) + pb('\033[byz\033[b', ('screen_repeat_character', 1), 'yz', ('screen_repeat_character', 1)) + # repeat 'x' at 3, then 'yz' at 4-5, then repeat 'z' at 6 + self.ae(str(s.line(0)), 'xxxyzzxx') + s.reset() + pb(' \033[3b', ' ', ('screen_repeat_character', 3)) + self.ae(str(s.line(0)), ' ') + s.reset() + pb('\t\033[b', ('screen_tab',), ('screen_repeat_character', 1)) + # The terminal includes blank tab padding in its line text. + self.ae(str(s.line(0)).rstrip(' '), '\t') + self.ae(s.cursor.x, 7) + self.ae(s.cursor.y, 0) + s.reset() + b']]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]' + + def test_osc_codes(self): + s = self.create_screen() + pb = partial(self.parse_bytes_dump, s) + c = s.callbacks + pb('a\033]2;x\\ryz\033\\bcde', 'a', ('set_title', 'x\\ryz'), 'bcde') + self.ae(str(s.line(0)), 'abcde') + self.ae(c.titlebuf, ['x\\ryz']) + c.clear() + pb('\033]\x07', ('set_title', ''), ('set_icon', '')) + self.ae(c.titlebuf, ['']), self.ae(c.iconbuf, '') + pb('1\033]ab\x072', '1', ('set_title', 'ab'), ('set_icon', 'ab'), '2') + self.ae(c.titlebuf, ['', 'ab']), self.ae(c.iconbuf, 'ab') + c.clear() + pb('\033]2;;;;\x07', ('set_title', ';;;')) + self.ae(c.titlebuf, [';;;']) + c.clear() + pb('\033]2;\x07', ('set_title', '')) + self.ae(c.titlebuf, ['']) + pb('\033]110\x07', ('set_dynamic_color', 110, '')) + self.ae(c.colorbuf, '') + c.clear() + pb('\033]9;\x07', ('desktop_notify', 9, '')) + pb('\033]9;test it with a nice long string\x07', ('desktop_notify', 9, 'test it with a nice long string')) + pb('\033]99;moo=foo;test it\x07', ('desktop_notify', 99, 'moo=foo;test it')) + self.ae(c.notifications, [(9, ''), (9, 'test it with a nice long string'), (99, 'moo=foo;test it')]) + c.clear() + pb('\033]8;;\x07', ('set_active_hyperlink', None, None)) + pb('\033]8moo\x07', ('Ignoring malformed OSC 8 code',)) + pb('\033]8;moo\x07', ('Ignoring malformed OSC 8 code',)) + pb('\033]8;id=xyz;\x07', ('set_active_hyperlink', 'xyz', None)) + pb('\033]8;moo:x=z:id=xyz:id=abc;http://yay;.com\x07', ('set_active_hyperlink', 'xyz', 'http://yay;.com')) + c.clear() + payload = '1' * 1024 + pb(f'\033]52;p;{payload}\x07', ('clipboard_control', 52, f'p;{payload}')) + c.clear() + pb('\033]52;p;xyz\x07', ('clipboard_control', 52, 'p;xyz')) + c.clear() + pb('\033]22;?__current__\x07', ('set_dynamic_color', 22, '?__current__')) + + def test_dcs_codes(self): + s = self.create_screen() + c = s.callbacks + pb = partial(self.parse_bytes_dump, s) + q = hexlify(b'kind').decode('ascii') + pb(f'a\033P+q{q}\033\\bcde', 'a', ('screen_request_capabilities', 43, q), 'bcde') + self.ae(str(s.line(0)), 'abcde') + self.ae(c.wtcbuf, '\033P1+r{}={}\033\\'.format(q, '1b5b313b3242').encode('ascii')) + c.clear() + pb('\033P$q q\033\\', ('screen_request_capabilities', ord('$'), ' q')) + self.ae(c.wtcbuf, b'\033P1$r1 q\033\\') + c.clear() + pb('\033P$qm\033\\', ('screen_request_capabilities', ord('$'), 'm')) + self.ae(c.wtcbuf, b'\033P1$rm\033\\') + for sgr in '0;34;102;1;2;3;4 0;38:5:200;58:2:10:11:12'.split(): + expected = set(sgr.split(';')) + c.clear() + parse_bytes(s, f'\033[{sgr}m\033P$qm\033\\'.encode('ascii')) + r = c.wtcbuf.decode('ascii').partition('r')[2].partition('m')[0] + self.ae(expected, set(r.split(';'))) + c.clear() + pb('\033P$qr\033\\', ('screen_request_capabilities', ord('$'), 'r')) + self.ae(c.wtcbuf, f'\033P1$r{s.margin_top + 1};{s.margin_bottom + 1}r\033\\'.encode('ascii')) + pb('\033P@kitty-cmd{abc\033\\', ('handle_remote_cmd', '{abc')) + p = base64_encode('abcd').decode() + pb(f'\033P@kitty-print|{p}\033\\', ('handle_remote_print', p)) + self.ae(['abcd'], s.callbacks.printbuf) + + c.clear() + pb('\033[?2026$p', ('report_mode_status', 2026, 1)) + self.ae(c.wtcbuf, b'\x1b[?2026;2$y') + pb('\033P=1s\033\\', ('screen_start_pending_mode',)) + c.clear() + pb('\033[?2026$p', ('report_mode_status', 2026, 1)) + self.ae(c.wtcbuf, b'\x1b[?2026;1$y') + pb('\033P=2s\033\\', ('screen_stop_pending_mode',)) + c.clear() + pb('\033[?2026$p', ('report_mode_status', 2026, 1)) + self.ae(c.wtcbuf, b'\x1b[?2026;2$y') + + + def test_oth_codes(self): + s = self.create_screen() + pb = partial(self.parse_bytes_dump, s) + pb('a\033_+\\+\033\\bcde', ('draw', 'a'), ('Unrecognized APC code: 0x2b',), ('draw', 'bcde')) + pb('a\033^+\\+\033\\bcde', ('draw', 'a'), ('Unrecognized PM code: 0x2b',), ('draw', 'bcde')) + pb('a\033X+\\+\033\\bcde', ('draw', 'a'), ('Unrecognized SOS code: 0x2b',), ('draw', 'bcde')) + + def test_graphics_command(self): + from base64 import standard_b64encode + + def enc(x): + return standard_b64encode(x.encode('utf-8') if isinstance(x, str) else x).decode('ascii') + + def c(**k): + for p, v in tuple(k.items()): + if isinstance(v, str) and p != 'payload': + k[p] = v.encode('ascii') + for f in 'action delete_action transmission_type compressed'.split(): + k.setdefault(f, b'\0') + for f in ('format more id data_sz data_offset width height x_offset y_offset data_height data_width cursor_movement' + ' num_cells num_lines cell_x_offset cell_y_offset z_index placement_id image_number quiet unicode_placement' + ' parent_id parent_placement_id offset_from_parent_x offset_from_parent_y' + ).split(): + k.setdefault(f, 0) + p = k.pop('payload', '') + k[''] = p + return ('graphics_command', k) + + def t(cmd, **kw): + pb('\033_G{};{}\033\\'.format(cmd, enc(kw.get('payload', ''))), c(**kw)) + + def e(cmd, err): + pb(f'\033_G{cmd}\033\\', (err,)) + + s = self.create_screen() + pb = partial(self.parse_bytes_dump, s) + uint32_max = 2**32 - 1 + t('i=%d' % uint32_max, id=uint32_max) + t('i=3,p=4', id=3, placement_id=4) + e('i=%d' % (uint32_max + 1), 'Malformed GraphicsCommand control block, number is too large') + pb('\033_Gi=12\033\\', c(id=12)) + t('a=t,t=d,s=100,z=-9', payload='X', action='t', transmission_type='d', data_width=100, z_index=-9) + t('a=t,t=d,s=100,z=9', payload='payload', action='t', transmission_type='d', data_width=100, z_index=9) + t('a=t,t=d,s=100,z=9,q=2', action='t', transmission_type='d', data_width=100, z_index=9, quiet=2) + e(',s=1', 'Malformed GraphicsCommand control block, invalid key character: 0x2c') + e('W=1', 'Malformed GraphicsCommand control block, invalid key character: 0x57') + e('1=1', 'Malformed GraphicsCommand control block, invalid key character: 0x31') + e('a=t,,w=2', 'Malformed GraphicsCommand control block, invalid key character: 0x2c') + e('s', 'Malformed GraphicsCommand control block, no = after key') + e('s=', 'Malformed GraphicsCommand control block, expecting an integer value') + e('s==', 'Malformed GraphicsCommand control block, expecting an integer value for key: s') + e('s=1=', 'Malformed GraphicsCommand control block, expecting a , or semi-colon after a value, found: 0x3d') + + def test_deccara(self): + s = self.create_screen() + pb = partial(self.parse_bytes_dump, s) + pb('\033[$r', ('deccara', '0;0;0;0;0')) + pb('\033[;;;;4:3;38:5:10;48:2:1:2:3;1$r', + ('deccara', '0;0;0;0;4:3'), ('deccara', '0;0;0;0;38:5:10'), ('deccara', '0;0;0;0;48:2:1:2:3'), ('deccara', '0;0;0;0;1')) + for y in range(s.lines): + line = s.line(y) + for x in range(s.columns): + c = line.cursor_from(x) + self.ae(c.bold, True) + self.ae(c.italic, False) + self.ae(c.decoration, 3) + self.ae(c.fg, (10 << 8) | 1) + self.ae(c.bg, (1 << 24 | 2 << 16 | 3 << 8 | 2)) + self.ae(s.line(0).cursor_from(0).bold, True) + pb('\033[1;2;2;3;22;39$r', ('deccara', '1;2;2;3;22;39')) + self.ae(s.line(0).cursor_from(0).bold, True) + line = s.line(0) + for x in range(1, s.columns): + c = line.cursor_from(x) + self.ae(c.bold, False) + self.ae(c.fg, 0) + line = s.line(1) + for x in range(0, 3): + c = line.cursor_from(x) + self.ae(c.bold, False) + self.ae(line.cursor_from(3).bold, True) + pb('\033[2*x\033[3;2;4;3;34$r\033[*x', ('screen_decsace', 2), ('deccara', '3;2;4;3;34'), ('screen_decsace', 0)) + for y in range(2, 4): + line = s.line(y) + for x in range(s.columns): + self.ae(line.cursor_from(x).fg, (10 << 8 | 1) if x < 1 or x > 2 else (4 << 8) | 1) + + +if __name__ == '__main__': + import unittest + unittest.main() diff --git a/tests/extern/kitty/screen.py b/tests/extern/kitty/screen.py new file mode 100644 index 0000000..80f8e71 --- /dev/null +++ b/tests/extern/kitty/screen.py @@ -0,0 +1,476 @@ +"""ctypes wrapper providing the Kitty ``Screen`` API on top of our C terminal. + +The shared library ``libterm.so`` is loaded via :mod:`ctypes`. Python calls +only the ``kitty_*`` functions of ``screen_bridge.c``, which wrap the public +API (declared in ``src/term/term.h``). Python classes on top of them match the +subset of Kitty's test API used by ``parser.py``. +""" + +import ctypes +import os + +# --------------------------------------------------------------------------- +# Shared library loading +# --------------------------------------------------------------------------- + +_lib = ctypes.CDLL(os.environ.get('WSAP_KITTY_LIB', os.path.join(os.path.dirname(__file__), 'libterm.so'))) + +# struct term_cell { uint32_t text, fg, bg; uint16_t attrs; uint8_t width, reserved; } +class TermCell(ctypes.Structure): + _fields_ = [ + ('text', ctypes.c_uint32), + ('fg', ctypes.c_uint32), + ('bg', ctypes.c_uint32), + ('attrs', ctypes.c_uint16), + ('width', ctypes.c_uint8), + ('reserved', ctypes.c_uint8), + ] + + +assert ctypes.sizeof(TermCell) == 16 + +# Reply callback type: void (*)(void *ud, const char *bytes, size_t len). +# The bytes are not NUL-terminated, so the callback takes a raw pointer. +REPLY_FN = ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(ctypes.c_char), ctypes.c_size_t) + +# Function signatures +_lib.kitty_term_new.restype = ctypes.c_void_p +_lib.kitty_term_new.argtypes = [ctypes.c_int, ctypes.c_int, REPLY_FN, ctypes.c_void_p] +_lib.kitty_term_free.restype = None +_lib.kitty_term_free.argtypes = [ctypes.c_void_p] +_lib.kitty_term_resize.restype = ctypes.c_bool +_lib.kitty_term_resize.argtypes = [ctypes.c_void_p, ctypes.c_int, ctypes.c_int] +_lib.kitty_term_write.restype = None +_lib.kitty_term_write.argtypes = [ctypes.c_void_p, ctypes.c_char_p, ctypes.c_size_t] +_lib.kitty_term_cell.restype = ctypes.c_bool +_lib.kitty_term_cell.argtypes = [ctypes.c_void_p, ctypes.c_int, ctypes.c_int, + ctypes.POINTER(TermCell)] +_lib.kitty_term_cell_text.restype = ctypes.c_int +_lib.kitty_term_cell_text.argtypes = [ctypes.c_void_p, ctypes.c_int, ctypes.c_int, + ctypes.POINTER(ctypes.c_uint32), ctypes.c_int] +for _name, _type in { + 'cols': ctypes.c_int, + 'rows': ctypes.c_int, + 'cursor_x': ctypes.c_int, + 'cursor_y': ctypes.c_int, + 'cursor_style': ctypes.c_int, + 'cursor_blink': ctypes.c_bool, + 'application_cursor': ctypes.c_bool, +}.items(): + _fn = getattr(_lib, 'kitty_term_' + _name) + _fn.argtypes = [ctypes.c_void_p] + _fn.restype = _type + +# --------------------------------------------------------------------------- +# Constants +# --------------------------------------------------------------------------- + +TERM_DEFAULT_FG = 0xFFFFFFFF +TERM_DEFAULT_BG = 0xFF1E1E2E + +# Cell attributes (must match enum term_attr in term.h) +TERM_ATTR_BOLD = 1 << 0 +TERM_ATTR_ITALIC = 1 << 1 +TERM_ATTR_FAINT = 1 << 2 +TERM_ATTR_STRIKE = 1 << 3 +TERM_ATTR_OVERLINE = 1 << 4 +TERM_ATTR_LINK = 1 << 5 +TERM_ATTR_EMOJI = 1 << 6 +TERM_ATTR_TEXT = 1 << 7 +TERM_ATTR_UNDERLINE_SHIFT = 8 +TERM_ATTR_UNDERLINE_MASK = 7 << TERM_ATTR_UNDERLINE_SHIFT + +# The longest grapheme cluster that a cell's text is read with. +_CLUSTER_CAP = 32 + +# Test-only accessors compiled against the current C terminal definition. +for _name, _type in { + 'scroll_top': ctypes.c_int, + 'scroll_bottom': ctypes.c_int, + 'cur_fg': ctypes.c_uint32, + 'cur_bg': ctypes.c_uint32, + 'bold': ctypes.c_bool, + 'faint': ctypes.c_bool, + 'italic': ctypes.c_bool, + 'strike': ctypes.c_bool, + 'inverse': ctypes.c_bool, + 'underline': ctypes.c_uint8, +}.items(): + _fn = getattr(_lib, 'kitty_screen_' + _name) + _fn.argtypes = [ctypes.c_void_p] + _fn.restype = _type + +_lib.kitty_screen_row_length.argtypes = [ctypes.c_void_p, ctypes.c_int] +_lib.kitty_screen_row_length.restype = ctypes.c_int +_lib.kitty_screen_cell_is_tab.argtypes = [ctypes.c_void_p, ctypes.c_int, ctypes.c_int] +_lib.kitty_screen_cell_is_tab.restype = ctypes.c_bool + +# ANSI color table from escape.c — used for reverse-mapping to Kitty indices. +# Must match the built-in default palette in src/term/palette.c. +_ANSI_COLORS = [ + 0xFF585B70, 0xFFCC6666, 0xFFB5BD68, 0xFFF0C674, + 0xFF81A2BE, 0xFFB294BB, 0xFF8ABEB7, 0xFFC5C8C6, + 0xFF666666, 0xFFD54E53, 0xFFB9CA4A, 0xFFE7C547, + 0xFF7AA6DA, 0xFFC397D8, 0xFF70C0B1, 0xFFEAEAEA, +] + + +def _build_color_table(): + """Build a reverse lookup from RGB (0xFFRRGGBB) to Kitty indexed color.""" + table = {} + for i, rgb in enumerate(_ANSI_COLORS): + table[rgb] = (i << 8) | 1 + levels = [0, 95, 135, 175, 215, 255] + for i in range(16, 232): + n = i - 16 + r, g, b = levels[n // 36], levels[(n // 6) % 6], levels[n % 6] + rgb = 0xFF000000 | (r << 16) | (g << 8) | b + table[rgb] = (i << 8) | 1 + for i in range(232, 256): + gray = 8 + (i - 232) * 10 + rgb = 0xFF000000 | (gray << 16) | (gray << 8) | gray + table[rgb] = (i << 8) | 1 + return table + + +_COLOR_TO_KITTY = _build_color_table() + + +TERM_COLOR_REF = 0x01000000 + + +def _convert_color(color): + """Convert our cell color to Kitty's format. + + Ours: a palette reference (top byte 0x01; slot 0-255, 0x100 default fg, + 0x101 default bg) or a literal 0xFFRRGGBB color. + + Kitty format: + 0 = default + index<<8|1 = indexed (0-255) + r<<24|g<<16|b<<8|2 = RGB + """ + if (color >> 24) == (TERM_COLOR_REF >> 24): + slot = color & 0x1FF + return 0 if slot >= 0x100 else (slot << 8) | 1 + if color == TERM_DEFAULT_FG or color == TERM_DEFAULT_BG: + return 0 + kitty = _COLOR_TO_KITTY.get(color) + if kitty is not None: + return kitty + r = (color >> 16) & 0xFF + g = (color >> 8) & 0xFF + b = color & 0xFF + return (r << 24) | (g << 16) | (b << 8) | 2 + + +# --------------------------------------------------------------------------- +# Screen property readers +# --------------------------------------------------------------------------- + +def _read_property(ptr, name): + return getattr(_lib, 'kitty_screen_' + name)(ptr) + + +# --------------------------------------------------------------------------- +# Cursor — wraps the terminal's current cursor SGR state + position +# --------------------------------------------------------------------------- + +class Cursor: + """Wraps the terminal's current cursor state (SGR attributes + position).""" + + def __init__(self, screen): + self._screen = screen + + @property + def _ptr(self): + return self._screen._ptr + + # -- position -- + @property + def x(self): + return _lib.kitty_term_cursor_x(self._screen._term) + + @property + def y(self): + return _lib.kitty_term_cursor_y(self._screen._term) + + # -- SGR boolean attributes -- + @property + def bold(self): + return _read_property(self._ptr, 'bold') + + @property + def italic(self): + return _read_property(self._ptr, 'italic') + + @property + def dim(self): + return _read_property(self._ptr, 'faint') + + @property + def reverse(self): + return _read_property(self._ptr, 'inverse') + + @property + def strikethrough(self): + return _read_property(self._ptr, 'strike') + + @property + def decoration(self): + # Kitty: 0=none, 1=straight, 2=double, 3=curly, 4=dotted, 5=dashed + return _read_property(self._ptr, 'underline') + + # -- colors -- + @property + def fg(self): + return _convert_color(_read_property(self._ptr, 'cur_fg')) + + @property + def bg(self): + return _convert_color(_read_property(self._ptr, 'cur_bg')) + + # -- cursor style -- + @property + def blink(self): + return _lib.kitty_term_cursor_blink(self._screen._term) + + @property + def shape(self): + return _lib.kitty_term_cursor_style(self._screen._term) + + +# --------------------------------------------------------------------------- +# CellCursor — wraps a single cell's attributes +# --------------------------------------------------------------------------- + +class CellCursor: + """Wraps a single cell's attributes for SGR inspection.""" + + def __init__(self, cell): + self._cell = cell + + @property + def bold(self): + return bool(self._cell.attrs & TERM_ATTR_BOLD) + + @property + def italic(self): + return bool(self._cell.attrs & TERM_ATTR_ITALIC) + + @property + def dim(self): + return bool(self._cell.attrs & TERM_ATTR_FAINT) + + @property + def reverse(self): + # Our terminal swaps fg/bg when inverse is set, so cells don't + # carry a separate inverse flag. + return False + + @property + def strikethrough(self): + return bool(self._cell.attrs & TERM_ATTR_STRIKE) + + @property + def decoration(self): + # enum term_underline uses Kitty's numbers: 0=none, 1=straight, + # 2=double, 3=curly, 4=dotted, 5=dashed. + return (self._cell.attrs & TERM_ATTR_UNDERLINE_MASK) >> TERM_ATTR_UNDERLINE_SHIFT + + @property + def fg(self): + return _convert_color(self._cell.fg) + + @property + def bg(self): + return _convert_color(self._cell.bg) + + +# --------------------------------------------------------------------------- +# Line — wraps a screen line, providing cell access +# --------------------------------------------------------------------------- + +class Line: + """Wraps a screen line, providing cell access and text content.""" + + def __init__(self, screen, y): + self._screen = screen + self._y = y + + def cursor_from(self, x): + """Return a :class:`CellCursor` for cell (x, y).""" + cell = TermCell() + if not _lib.kitty_term_cell(self._screen._term, x, self._y, ctypes.byref(cell)): + return CellCursor(TermCell()) + return CellCursor(cell) + + def __str__(self): + """Return the text content of the line up to the written length.""" + length = min(_lib.kitty_screen_row_length(self._screen._term, self._y), + self._screen.columns) + if length == 0: + return '' + result = [] + cell = TermCell() + cps = (ctypes.c_uint32 * _CLUSTER_CAP)() + for x in range(length): + if not _lib.kitty_term_cell(self._screen._term, x, self._y, ctypes.byref(cell)): + result.append(' ') + continue + # Width 0 is the right half of a wide glyph. + if cell.width == 0: + continue + if _lib.kitty_screen_cell_is_tab(self._screen._term, x, self._y): + result.append('\t') + continue + n = _lib.kitty_term_cell_text(self._screen._term, x, self._y, cps, _CLUSTER_CAP) + n = min(n, _CLUSTER_CAP) + result.append(''.join(chr(cp) for cp in cps[:n])) + return ''.join(result) + + +# --------------------------------------------------------------------------- +# Callbacks — mock callbacks object for OSC/DCS responses +# --------------------------------------------------------------------------- + +class Callbacks: + """Mock callbacks object for OSC/DCS responses.""" + + def __init__(self, screen=None): + self._screen = screen + self.wtcbuf = b'' + self.titlebuf = [] + self.iconbuf = '' + self.colorbuf = '' + self.notifications = [] + self.printbuf = [] + + def clear(self): + self.wtcbuf = b'' + self.titlebuf = [] + self.iconbuf = '' + self.colorbuf = '' + self.notifications = [] + self.printbuf = [] + if self._screen is not None: + self._screen._response_buf.clear() + + +# --------------------------------------------------------------------------- +# Screen — main screen wrapper using ctypes +# --------------------------------------------------------------------------- + +class Screen: + """Main screen wrapper using ctypes to drive the C terminal.""" + + def __init__(self, cols=80, rows=5): + self._cols = cols + self._rows = rows + self._response_buf = bytearray() + self._write_buf = bytearray() + self._callbacks = Callbacks(self) + # Keep a reference to the callback function so it isn't GC'd + self._resp_fn = REPLY_FN(self._on_response) + self._term = _lib.kitty_term_new(cols, rows, self._resp_fn, None) + if not self._term: + raise RuntimeError('kitty_term_new failed') + + @property + def _ptr(self): + return ctypes.cast(self._term, ctypes.c_void_p).value + + # -- response callback -- + def _on_response(self, ud, data, length): + if data and length: + self._response_buf.extend(data[:length]) + self._callbacks.wtcbuf = bytes(self._response_buf) + + # -- writing -- + def write(self, data): + if data: + _lib.kitty_term_write(self._term, bytes(data), len(data)) + + # -- test write buffer API (matches Kitty's test_parse interface) -- + def test_create_write_buffer(self): + """Return the available space in the write buffer (int, falsy when full).""" + from fast_data_types import VT_PARSER_BUFFER_SIZE + return VT_PARSER_BUFFER_SIZE - len(self._write_buf) + + def test_commit_write_buffer(self, data, write_buf): + """Append *data* to the internal write buffer, return bytes consumed.""" + from fast_data_types import VT_PARSER_BUFFER_SIZE + available = VT_PARSER_BUFFER_SIZE - len(self._write_buf) + n = min(len(data), available) + if n > 0: + self._write_buf.extend(data[:n]) + return n + + def test_parse_written_data(self, cd=None): + """Parse the write buffer, dispatching to the C terminal and callback.""" + data = bytes(self._write_buf) + self._write_buf.clear() + from kitty_parser import parse_bytes + if cd is not None: + parse_bytes(self, data, cd) + elif data: + # No callback — still parse to keep parser state in sync + parse_bytes(self, data, lambda *a: None) + + # -- properties -- + @property + def cursor(self): + return Cursor(self) + + @property + def callbacks(self): + return self._callbacks + + @property + def columns(self): + return _lib.kitty_term_cols(self._term) + + @property + def lines(self): + return _lib.kitty_term_rows(self._term) + + @property + def margin_top(self): + return _read_property(self._ptr, 'scroll_top') + + @property + def margin_bottom(self): + return _read_property(self._ptr, 'scroll_bottom') + + @property + def cursor_key_mode(self): + return _lib.kitty_term_application_cursor(self._term) + + @cursor_key_mode.setter + def cursor_key_mode(self, value): + if value: + self.write(b'\x1b[?1h') + else: + self.write(b'\x1b[?1l') + + # -- line access -- + def line(self, n): + return Line(self, n) + + # -- operations -- + def reset(self): + self.write(b'\x1bc') + + def cursor_move(self, n): + """Move cursor back n cells (kitty cursor_move: direction -1, + allowed to move to previous lines).""" + self.write(b'\x1b[' + str(n).encode() + b'D') + + def close(self): + """Destroy the underlying C terminal.""" + if getattr(self, '_term', None): + _lib.kitty_term_free(self._term) + self._term = None + + def __del__(self): + self.close() diff --git a/tests/extern/kitty/screen_bridge.c b/tests/extern/kitty/screen_bridge.c new file mode 100644 index 0000000..63a3583 --- /dev/null +++ b/tests/extern/kitty/screen_bridge.c @@ -0,0 +1,113 @@ +/* Test-only entry points for the Kitty screen wrapper in screen.py. + * Python calls only these functions, so it needs no struct layouts other + * than the public struct term_cell. The kitty_screen_* accessors read + * properties that term.h does not expose. They compile against the current + * terminal layout rather than duplicating byte offsets in Python. */ +#include "internal.h" +#include "screen.h" + +typedef void (*kitty_reply_fn)(void *ud, const char *bytes, size_t len); + +struct term *kitty_term_new(int cols, int rows, kitty_reply_fn reply, void *ud) +{ + return term_new(&(struct term_options){ + .size = { .x = cols, .y = rows }, + .callbacks = { .ud = ud, .reply = reply }, + }); +} + +/* One snapshot serves every cell read. */ +static struct term_snapshot *snap; + +void kitty_term_free(struct term *t) +{ + /* A new terminal can get the address of the freed one. The snapshot + * would then keep its rows, so it goes away too. */ + term_snapshot_free(snap); + snap = NULL; + term_free(t); +} + +void kitty_term_write(struct term *t, const char *bytes, size_t len) +{ + term_feed(t, bytes, len, 0); +} + +bool kitty_term_resize(struct term *t, int cols, int rows) +{ + return term_resize(t, (struct term_vector2i){ .x = cols, .y = rows }); +} + +int kitty_term_cols(struct term *t) { return term_size(t).x; } +int kitty_term_rows(struct term *t) { return term_size(t).y; } +int kitty_term_cursor_x(struct term *t) { return term_cursor(t).pos.x; } +int kitty_term_cursor_y(struct term *t) { return term_cursor(t).pos.y; } +int kitty_term_cursor_style(struct term *t) { return (int)term_cursor(t).style; } +bool kitty_term_cursor_blink(struct term *t) { return term_cursor(t).blink; } +bool kitty_term_application_cursor(struct term *t) { return term_modes(t).app_cursor; } + +/* Update the snapshot from the live screen, as a renderer reads it. + * Returns NULL when the cell is out of range or memory runs out. */ +static struct term_snapshot *cell_snapshot(struct term *t, int col, int row) +{ + struct term_vector2i size = term_size(t); + if (col < 0 || col >= size.x || row < 0 || row >= size.y) + return NULL; + if (!snap) + snap = term_snapshot_new(); + term_scroll_to(t, 0); + if (!snap || !term_snapshot_update(snap, t)) + return NULL; + return snap; +} + +bool kitty_term_cell(struct term *t, int col, int row, struct term_cell *out) +{ + struct term_snapshot *s = cell_snapshot(t, col, row); + if (!s) + return false; + *out = term_snapshot_row(s, row)[col]; + return true; +} + +/* The code points of a cell's text: one, or a whole grapheme cluster. + * Returns the number of code points and writes at most cap of them. */ +int kitty_term_cell_text(struct term *t, int col, int row, uint32_t *cps, int cap) +{ + struct term_snapshot *s = cell_snapshot(t, col, row); + if (!s) + return 0; + return term_snapshot_cluster(s, term_snapshot_row(s, row)[col].text, cps, cap); +} + +#define PROPERTY(type, name) \ + type kitty_screen_##name(struct term *t) { return t->name; } + +PROPERTY(int, scroll_top) +PROPERTY(int, scroll_bottom) +PROPERTY(uint32_t, cur_fg) +PROPERTY(uint32_t, cur_bg) +PROPERTY(bool, bold) +PROPERTY(bool, faint) +PROPERTY(bool, italic) +PROPERTY(bool, strike) +PROPERTY(bool, inverse) +PROPERTY(uint8_t, underline) + +int kitty_screen_row_length(struct term *t, int row) +{ + if (!t || row < 0 || row >= t->rows) return 0; + const struct srow *r = term_screen_row(t, row); + return r ? (int)r->len : 0; +} + +/* A tab stores a marker cell that a snapshot shows as a blank. Kitty keeps + * the tab in its line text, so the wrapper asks for the marker. */ +bool kitty_screen_cell_is_tab(struct term *t, int col, int row) +{ + if (col < 0 || col >= t->cols || row < 0 || row >= t->rows) + return false; + term_scroll_to(t, 0); + const struct cell *cells = term_screen_view_row(t, row); + return cells && cells[col].c == '\t'; +} diff --git a/tests/fuzz_gen.h b/tests/fuzz_gen.h new file mode 100644 index 0000000..166d729 --- /dev/null +++ b/tests/fuzz_gen.h @@ -0,0 +1,188 @@ +#ifndef WSAP_TERM_FUZZ_GEN_H +#define WSAP_TERM_FUZZ_GEN_H + +/* + * The input generator that write-fuzz-test and model-test share. gen() + * makes whole sequences, malformed UTF-8, and raw bytes. The model handles + * every byte that gen() makes. + */ + +#include <stdarg.h> +#include <stdint.h> +#include <stdio.h> +#include <string.h> + +static uint64_t rng; + +static uint32_t rnd(void) +{ + rng ^= rng << 13; + rng ^= rng >> 7; + rng ^= rng << 17; + return (uint32_t)rng; +} + +static int rn(int n) +{ + return (int)(rnd() % (uint32_t)n); +} + +static char buf[1 << 16]; +static size_t blen; + +static void putb(char c) +{ + if (blen + 1 < sizeof(buf)) + buf[blen++] = c; +} + +static void put(const char *s) +{ + size_t n = strlen(s); + if (blen + n < sizeof(buf)) { + memcpy(buf + blen, s, n); + blen += n; + } +} + +static void __attribute__((format(printf, 1, 2))) putf(const char *fmt, ...) +{ + char tmp[64]; + va_list args; + va_start(args, fmt); + vsnprintf(tmp, sizeof(tmp), fmt, args); + va_end(args); + put(tmp); +} + +#define PICK(list) (list)[rn((int)(sizeof(list) / sizeof((list)[0])))] + +static void gen_sgr(void) +{ + static const int plain[] = { + 0, 1, 2, 3, 4, 5, 7, 8, 9, 21, 22, 23, 24, 27, 29, 39, 49, 53, 55, 59, + }; + put("\033["); + int n = 1 + rn(3); + for (int i = 0; i < n; i++) { + if (i) put(";"); + switch (rn(9)) { + case 0: putf("38;5;%d", rn(300)); break; + case 1: putf("48;5;%d", rn(300)); break; + case 2: putf("38;2;%d;%d;%d", rn(256), rn(256), rn(256)); break; + case 3: putf("48:2::%d:%d:%d", rn(256), rn(256), rn(256)); break; + case 4: putf("4:%d", rn(7)); break; + case 5: putf("%d", 30 + rn(8) + (rn(2) ? 60 : 0)); break; + case 6: putf("%d", 40 + rn(8) + (rn(2) ? 60 : 0)); break; + case 7: putf("58;5;%d", rn(256)); break; + default: putf("%d", PICK(plain)); break; + } + } + put("m"); +} + +static inline void gen(int ops, int cols, int rows) +{ + static const char *words[] = { + "a", "hello", "x y", "0123456789", "日本", "é", "中文字", + "e\xcc\x81", "👍", " ", + }; + /* Include valid text, malformed UTF-8, and incomplete UTF-8. */ + static const char *utf8_3[] = { + "中", "文", "字", "€", "ｱ", "\xe2\x83\x90", "\xed\xa0\x80", + "\xe0\x80\xaf", "\xef\xbf\xbd", "한", "a", "\xe2\x80\x8d", "\xe3\x80", + }; + /* Emoji, flags, modifiers, ZWJ sequences, presentation selectors, + * stacked marks, and C1 controls in UTF-8 form. */ + static const char *clusters[] = { + "😀", "\xf0\x9f\x87\xa9\xf0\x9f\x87\xaa", "👍🏽", "👨👩👧", "❤️", "☺︎", + "a\xcc\x81\xcc\x82", "\xe2\x80\x8b", "\xc2\x85", "\xc2\x84", "\xc2\x8d", + "\xc2\x88", "\xf0\x9f\x87\xa9", "\xcc\x81", + }; + static const char *modes[] = { + "\033[4h", "\033[4l", "\033[?7l", "\033[?7h", + "\033[?1049h", "\033[?1049l", "\033[?6h", "\033[?6l", + "\033[20h", "\033[20l", "\033[?25l", "\033[?25h", + "\033[?5h", "\033[?5l", + }; + static const char *escs[] = { + "\033D", "\033M", "\033E", "\0337", "\0338", "\033[s", "\033[u", "\033c", + "\033[?1048h", "\033[?1048l", "\033H", "\033=", "\033>", "\033%G", + }; + static const char *charsets[] = { + "\033(0", "\033(B", "\033)0", "\033)B", "\x0e", "\x0f", + }; + static const char *queries[] = { + "\033[5n", "\033[6n", "\033[c", "\033[0c", "\033[>c", "\033[1c", + "\033[14t", "\033P$qm\033\\", "\033P$qr\033\\", "\033[?6$p", + "\033[4$p", "\033[20$p", + }; + /* C0 controls inside CSI, cancelled sequences, and ignored strings. */ + static const char *odd[] = { + "\033[2\n;3H", "\033[1\bC", "\033[3\x18m", "\033[1\x1a;2H", + "\033[\1772C", "\x0b", "\x0c", "\x7f", "\x18", "\x1a", + "\033]0;title\007", "\033]2;x\033\\", "\033_apc\033\\", "\033^pm\007", + "\033X\033\\", "\033]133;A\007", "\033]133;A\033\\", "\033]133;D\007", + "\033#8", + }; + static const char csi_final[] = "ABCDEFGHIJKLMPSTXZ@`abdefghlmnrsu"; + + for (int i = 0; i < ops; i++) { + switch (rn(36)) { + case 0: case 1: case 2: put(PICK(words)); break; + case 3: { + int n = rn(cols * 2 + 2); + for (int k = 0; k < n; k++) + putb((char)(0x21 + rn(94))); + break; + } + case 4: put("\r\n"); break; + case 5: put("\n"); break; + case 6: put("\r"); break; + case 7: put("\t"); break; + case 8: put("\b"); break; + case 9: putf("\033[%d;%dH", rn(rows + 2), rn(cols + 2)); break; + case 10: putf("\033[%d%c", rn(4), "JK"[rn(2)]); break; + case 11: putf("\033[%d%c", rn(cols), "XP@LMST"[rn(7)]); break; + case 12: putf("\033[%d;%dm", 30 + rn(8), 40 + rn(8)); break; + case 13: put(rn(2) ? "\033[m" : "\033[0m"); break; + case 14: putf("\033[48;2;%d;%d;1m", rn(256), rn(256)); break; + case 15: putf("\033[%d%c", rn(cols + 2), csi_final[rn((int)sizeof(csi_final) - 1)]); break; + case 16: putf("\033[%d;%dr", rn(rows) + 1, rn(rows) + 1); break; + case 17: put(PICK(modes)); break; + case 18: put(PICK(escs)); break; + case 19: putf("\033[%d%c", rn(5), "IZb"[rn(3)]); break; + case 20: putf("\033[%d;%d;1;1$r", rn(rows) + 1, rn(cols) + 1); break; + case 21: put("\033[1;7m"); break; + case 22: case 23: { + /* Keep ESC out of raw bytes so later input reaches the fast paths. */ + int n = 64 + rn(400); + for (int k = 0; k < n; k++) { + unsigned char b = (unsigned char)rnd(); + putb((char)(b == 0x1b ? 'e' : b)); + } + break; + } + case 24: case 25: { + int n = 4 + rn(40); + for (int k = 0; k < n; k++) + put(utf8_3[rn(20) < 14 ? rn(3) : rn(13)]); + break; + } + case 26: case 27: gen_sgr(); break; + case 28: put(PICK(clusters)); break; + case 29: put(PICK(charsets)); break; + case 30: put(PICK(queries)); break; + case 31: put(PICK(odd)); break; + case 32: putf("\033[%d q", rn(7)); break; + case 33: putf("\033[%dg", rn(4)); break; + case 34: + putf("\033[%d;%d;%d;%d;%d$r", rn(rows) + 1, rn(cols) + 1, + rn(rows) + 1, rn(cols) + 1, PICK(((const int[]){ 0, 1, 4, 7, 22, 24, 27, 31, 42 }))); + break; + case 35: putf("\033[%d*x", rn(3)); break; + } + } +} + +#endif diff --git a/tests/gstore_test.c b/tests/gstore_test.c new file mode 100644 index 0000000..637b136 --- /dev/null +++ b/tests/gstore_test.c @@ -0,0 +1,227 @@ +/* + * Tests for the per-terminal grapheme store (gstore). + * + * Verifies: + * - Two terminals do not share clusters: freeing one leaves the other intact. + * - Clusters that remain visible on screen survive GC sweeps. + * - More than 65,536 distinct clusters can be interned as old ones scroll + * out of history, without reaching any cap or exhausting memory. + * - Direct unit tests of intern, expand, deduplication, and sweep. + */ +#include "gstore.h" +#include "internal.h" +#include "term/term.h" +#include "term/unicode/utf8.h" + +#include <assert.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> + +static void test_direct_unit(void) +{ + struct gstore *g = term_gstore_new(NULL); + assert(g != NULL); + assert(term_gstore_count(g) == 0); + + /* Single code point returns the code point itself. */ + uint32_t single = 'A'; + assert(term_gstore_intern(g, &single, 1) == 'A'); + assert(term_gstore_count(g) == 0); + + /* Cluster returns a handle. */ + uint32_t c1[2] = { 'e', 0x0301 }; + uint32_t h1 = term_gstore_intern(g, c1, 2); + assert(grapheme_is_handle(h1)); + assert(term_gstore_count(g) == 1); + + /* Identical cluster deduplicates to the same handle. */ + uint32_t h1_dup = term_gstore_intern(g, c1, 2); + assert(h1_dup == h1); + assert(term_gstore_count(g) == 1); + + /* Different cluster gets a distinct handle. */ + uint32_t c2[2] = { 'a', 0x0300 }; + uint32_t h2 = term_gstore_intern(g, c2, 2); + assert(grapheme_is_handle(h2)); + assert(h2 != h1); + assert(term_gstore_count(g) == 2); + + /* Expand both handles. */ + uint32_t out[GRAPHEME_MAX_LEN] = {0}; + int n = term_gstore_expand(g, h1, out, GRAPHEME_MAX_LEN); + assert(n == 2); + assert(out[0] == 'e' && out[1] == 0x0301); + + n = term_gstore_expand(g, h2, out, GRAPHEME_MAX_LEN); + assert(n == 2); + assert(out[0] == 'a' && out[1] == 0x0300); + + /* Sweep with h1 marked and h2 unmarked. */ + term_gstore_mark(g, h1); + term_gstore_sweep(g, NULL); + assert(term_gstore_count(g) == 1); + + /* h1 is still valid. */ + n = term_gstore_expand(g, h1, out, GRAPHEME_MAX_LEN); + assert(n == 2); + assert(out[0] == 'e' && out[1] == 0x0301); + + /* h2 was reclaimed. */ + n = term_gstore_expand(g, h2, out, GRAPHEME_MAX_LEN); + assert(n == 1 && out[0] == 0xFFFD); + + /* New cluster can reuse the reclaimed slot. */ + uint32_t c3[3] = { 'o', 0x0308, 0x0301 }; + uint32_t h3 = term_gstore_intern(g, c3, 3); + assert(grapheme_is_handle(h3)); + assert(term_gstore_count(g) == 2); + n = term_gstore_expand(g, h3, out, GRAPHEME_MAX_LEN); + assert(n == 3); + assert(out[0] == 'o' && out[1] == 0x0308 && out[2] == 0x0301); + + term_gstore_free(g); +} + +static void test_two_terminals_isolation(void) +{ + struct term_options opt = { .size = { .x = 20, .y = 4 } }; + struct term *t1 = term_new(&opt); + struct term *t2 = term_new(&opt); + assert(t1 && t2); + + const char *cluster_text = "e\xcc\x81"; /* 'e' with acute */ + term_feed(t1, cluster_text, strlen(cluster_text), 0); + term_feed(t2, cluster_text, strlen(cluster_text), 0); + + /* Free terminal 1. */ + term_free(t1); + + /* Terminal 2 must still expand its cluster correctly. */ + struct term_snapshot *snap = term_snapshot_new(); + assert(snap); + assert(term_snapshot_update(snap, t2)); + + const struct term_cell *row = term_snapshot_row(snap, 0); + assert(row[0].text & TERM_TEXT_CLUSTER); + + uint32_t cps[GRAPHEME_MAX_LEN] = {0}; + int n = term_snapshot_cluster(snap, row[0].text, cps, GRAPHEME_MAX_LEN); + assert(n == 2); + assert(cps[0] == 'e' && cps[1] == 0x0301); + + term_snapshot_free(snap); + term_free(t2); +} + +static void test_screen_clusters_survive_sweep(void) +{ + struct term_options opt = { .size = { .x = 40, .y = 10 }, .scrollback = 10 }; + struct term *t = term_new(&opt); + assert(t); + + /* Write 5 clusters on the top row. */ + term_feed(t, "a\xcc\x80" "b\xcc\x81" "c\xcc\x82" "d\xcc\x83" "e\xcc\x84", 15, 0); + + /* Write ephemeral clusters on row 1, then overwrite them. */ + for (int i = 0; i < 50; i++) { + term_feed(t, "\033[2;1H\033[2K", 8, 0); + char buf[32]; + int len = snprintf(buf, sizeof(buf), "z\xcc\x80\xcc\x81%d", i); + term_feed(t, buf, (size_t)len, 0); + } + term_feed(t, "\033[2;1H\033[2K", 8, 0); + + /* Explicit sweep. The clusters on row 0 are on screen and must survive. */ + term_gstore_sweep(t->gstore, t); + + struct term_snapshot *snap = term_snapshot_new(); + assert(snap); + assert(term_snapshot_update(snap, t)); + + /* Row 0 of screen has "a\xcc\x80...". */ + const struct term_cell *row = term_snapshot_row(snap, 0); + assert(row[0].text & TERM_TEXT_CLUSTER); + + uint32_t cps[GRAPHEME_MAX_LEN] = {0}; + int n = term_snapshot_cluster(snap, row[0].text, cps, GRAPHEME_MAX_LEN); + assert(n == 2); + assert(cps[0] == 'a' && cps[1] == 0x0300); + + n = term_snapshot_cluster(snap, row[1].text, cps, GRAPHEME_MAX_LEN); + assert(n == 2); + assert(cps[0] == 'b' && cps[1] == 0x0301); + + n = term_snapshot_cluster(snap, row[2].text, cps, GRAPHEME_MAX_LEN); + assert(n == 2); + assert(cps[0] == 'c' && cps[1] == 0x0302); + + term_snapshot_free(snap); + term_free(t); +} + +static void test_more_than_65536_clusters(void) +{ + /* Bounded history so old lines scroll out. */ + struct term_options opt = { .size = { .x = 80, .y = 24 }, .scrollback = 10 }; + struct term *t = term_new(&opt); + assert(t); + + /* + * Generate 66,000 distinct clusters. + * Base 'a' (0x61) + mark1 (0x0300..0x034F, 80 options) + mark2 (0x0350..0x036F + 0xFE00..0xFE0F, 830 options) + * 256 * 260 = 66,560 distinct clusters. + */ + char buf[64]; + uint32_t total = 0; + const uint32_t target = 66000; + + for (uint32_t i = 0; total < target; i++) { + char base = (char)('a' + (i % 26)); + uint32_t cp1 = 0x0300 + ((i / 26) % 55); + uint32_t cp2 = 0x0340 + ((i / (26 * 55)) % 47); + + size_t len = 0; + buf[len++] = base; + len += (size_t)utf8_encode(cp1, buf + len); + len += (size_t)utf8_encode(cp2, buf + len); + buf[len++] = '\r'; + buf[len++] = '\n'; + + term_feed(t, buf, len, 0); + total++; + } + + assert(total >= 65536); + + /* Verify the latest cluster on screen reads back correctly. */ + struct term_snapshot *snap = term_snapshot_new(); + assert(snap); + assert(term_snapshot_update(snap, t)); + + struct term_cursor c = term_cursor(t); + int y = c.pos.y > 0 ? c.pos.y - 1 : 0; + const struct term_cell *row = term_snapshot_row(snap, y); + assert(row[0].text & TERM_TEXT_CLUSTER); + + uint32_t cps[GRAPHEME_MAX_LEN] = {0}; + int n = term_snapshot_cluster(snap, row[0].text, cps, GRAPHEME_MAX_LEN); + assert(n == 3); + + /* Live cluster count must be small (only bounded screen and history). */ + uint32_t count = term_gstore_count(t->gstore); + assert(count < 2000); + + term_snapshot_free(snap); + term_free(t); +} + +int main(void) +{ + test_direct_unit(); + test_two_terminals_isolation(); + test_screen_clusters_survive_sweep(); + test_more_than_65536_clusters(); + puts("PASS: gstore tests"); + return 0; +} diff --git a/tests/header_test.sh b/tests/header_test.sh new file mode 100644 index 0000000..6636418 --- /dev/null +++ b/tests/header_test.sh @@ -0,0 +1,16 @@ +#!/usr/bin/env bash +# A host includes term.h and nothing else. The header compiles alone as +# strict C11 and as C++, with only include/term on the include path. +set -euo pipefail +cd "${1:-$(dirname "$0")/..}" +cc=${CC:-cc} +cxx=${CXX:-c++} + +status=0 +echo '#include "term.h"' | "$cc" -std=c11 -Wall -Wextra -Wpedantic -Werror \ + -Iinclude/term -x c -fsyntax-only - || { echo "term.h: not valid C11" >&2; status=1; } +if command -v "$cxx" >/dev/null; then + echo '#include "term.h"' | "$cxx" -std=c++17 -Wall -Wextra -Wpedantic -Werror \ + -Iinclude/term -x c++ -fsyntax-only - || { echo "term.h: not valid C++" >&2; status=1; } +fi +exit $status diff --git a/tests/helpers/term_api.h b/tests/helpers/term_api.h new file mode 100644 index 0000000..bfa8d71 --- /dev/null +++ b/tests/helpers/term_api.h @@ -0,0 +1,182 @@ +#ifndef WSAP_TEST_TERM_API_H +#define WSAP_TEST_TERM_API_H + +/* + * Short forms of the public terminal API for tests. Cell reads go through + * a snapshot, the same way a renderer reads, so tests check only what the + * terminal shows. + */ + +#include "term/term.h" + +#include <stdbool.h> +#include <stddef.h> +#include <stdlib.h> +#include <string.h> + +/* The history size the tests assume. */ +#define TEST_SCROLLBACK 4096 + +static inline struct term *tt_new_opts(int cols, int rows, const struct term_callbacks *cb) +{ + struct term_options opt = { + .size = { .x = cols, .y = rows }, + .scrollback = TEST_SCROLLBACK, + }; + if (cb) + opt.callbacks = *cb; + return term_new(&opt); +} + +static inline struct term *tt_new(int cols, int rows) +{ + return tt_new_opts(cols, rows, NULL); +} + +static inline void tt_write(struct term *t, const char *s, size_t len) +{ + term_feed(t, s, len, 0); +} + +static inline void tt_puts(struct term *t, const char *s) +{ + term_feed(t, s, strlen(s), 0); +} + +static inline int tt_cols(const struct term *t) { return term_size(t).x; } +static inline int tt_rows(const struct term *t) { return term_size(t).y; } +static inline int tt_cursor_x(const struct term *t) { return term_cursor(t).pos.x; } +static inline int tt_cursor_y(const struct term *t) { return term_cursor(t).pos.y; } +static inline int tt_history(const struct term *t) { return term_scrollback(t).total; } +static inline int tt_view_offset(const struct term *t) { return term_scrollback(t).offset; } + +/* One snapshot serves every read. It follows whichever terminal a test + * reads last. */ +static struct term_snapshot *tt_snapshot_for(struct term *t) +{ + static struct term_snapshot *snap; + if (!snap) + snap = term_snapshot_new(); + term_snapshot_update(snap, t); + return snap; +} + +static const struct term_cell tt_blank_cell = { + .text = ' ', .fg = TERM_COLOR_FG, .bg = TERM_COLOR_BG, .width = 1, +}; + +/* A cell of the viewport. Out-of-range columns read as a blank cell. */ +static inline const struct term_cell *tt_view_cell(struct term *t, int col, int row) +{ + struct term_snapshot *s = tt_snapshot_for(t); + const struct term_cell *cells = term_snapshot_row(s, row); + if (!cells) + return NULL; + if (col < 0 || col >= term_snapshot_view(s)->size.x) + return &tt_blank_cell; + return &cells[col]; +} + +/* A cell of the live screen, whatever the viewport shows. */ +static inline const struct term_cell *tt_cell(struct term *t, int col, int row) +{ + int offset = tt_view_offset(t); + if (offset) + term_scroll_to(t, 0); + const struct term_cell *cell = tt_view_cell(t, col, row); + if (offset) + term_scroll_to(t, offset); + return cell; +} + +/* The code points of a cell's text: one, or a whole cluster. The snapshot + * that the last read used must still describe the terminal. */ +static inline int tt_cell_cps(struct term *t, const struct term_cell *cell, + uint32_t *cps, int cap) +{ + return term_snapshot_cluster(tt_snapshot_for(t), cell->text, cps, cap); +} + +/* The first code point of a cell's text. */ +static inline uint32_t tt_cell_cp(struct term *t, const struct term_cell *cell) +{ + uint32_t cp = 0; + if (!(cell->text & TERM_TEXT_CLUSTER)) + return cell->text; + tt_cell_cps(t, cell, &cp, 1); + return cp; +} + +static inline bool tt_is_cluster(const struct term_cell *cell) +{ + return (cell->text & TERM_TEXT_CLUSTER) != 0; +} + +/* The selected columns [begin, end) of a viewport row. */ +static inline struct term_span tt_row_selection(struct term *t, int row) +{ + return term_snapshot_row_selection(tt_snapshot_for(t), row); +} + +static inline bool tt_selected(struct term *t, int col, int row) +{ + struct term_span span = tt_row_selection(t, row); + return col >= span.begin && col < span.end; +} + +/* Select from one viewport cell to another, both inclusive. */ +static inline void tt_select(struct term *t, int x1, int y1, int x2, int y2) +{ + bool forward = y1 < y2 || (y1 == y2 && x1 <= x2); + struct term_vector2i a = { .x = x1, .y = y1 }, b = { .x = x2, .y = y2 }; + term_select_start(t, forward ? a : b, false, TERM_SELECT_CELL); + term_select_extend(t, forward ? b : a, true); +} + +/* The helpers below return new strings. The caller frees them. */ +/* NULL when nothing is selected. */ +static inline char *tt_selection_text(const struct term *t) +{ + size_t n = term_selection_text(t, NULL, 0); + if (!n) + return NULL; + char *text = malloc(n + 1); + if (text) + term_selection_text(t, text, n + 1); + return text; +} + +static inline char *tt_history_text(struct term *t, int max_rows) +{ + size_t n = term_history_text(t, max_rows, NULL, 0); + char *text = malloc(n + 1); + if (text) + term_history_text(t, max_rows, text, n + 1); + return text; +} + +/* NULL when there is no link. */ +static inline char *tt_link_at(struct term *t, int col, int row) +{ + struct term_vector2i cell = { .x = col, .y = row }; + size_t n = term_link_at(t, cell, NULL, 0); + if (!n) + return NULL; + char *url = malloc(n + 1); + if (url) + term_link_at(t, cell, url, n + 1); + return url; +} + +static inline bool tt_resize(struct term *t, int cols, int rows) +{ + return term_resize(t, (struct term_vector2i){ .x = cols, .y = rows }); +} + +/* The ARGB value of a color in the terminal's current palette. */ +static inline uint32_t tt_color(struct term *t, term_color c) +{ + return term_view_color(term_snapshot_view(tt_snapshot_for(t)), c); +} + +#endif diff --git a/tests/hist_test.c b/tests/hist_test.c new file mode 100644 index 0000000..be9fd3f --- /dev/null +++ b/tests/hist_test.c @@ -0,0 +1,765 @@ +#include "hist.h" +#include "hist_view.h" +#include "term/unicode/grapheme.h" + +#include <assert.h> +#include <limits.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> + +static uint64_t lines(const struct hist *h) +{ + return hist_end_id(h) - hist_first_id(h); +} + +static struct cell dc(uint32_t c) +{ + return (struct cell){ .c = c, .fg = TERM_COLOR_FG, .bg = TERM_COLOR_BG }; +} + +static void test_init_free(void) +{ + struct hist h; + assert(term_hist_init(&h, 100, 64 * 1024 * 4)); + assert(lines(&h) == 0); + assert(hist_first_id(&h) == 0); + assert(!hist_is_open(&h)); + term_hist_free(&h); +} + +static void test_ascii_roundtrip(void) +{ + struct hist h; + assert(term_hist_init(&h, 100, 64 * 1024 * 4)); + + const char *msg = "Hello, world!"; + term_hist_push_ascii(&h, msg, (int)strlen(msg), NULL, false); + assert(lines(&h) == 1); + assert(term_hist_line_rows(&h, 0, 80) == 1); + assert(term_hist_line_rows(&h, 0, 5) == 3); + + struct cell out[80]; + int n = term_hist_decode_segment(&h, 0, 0, 80, out, NULL); + assert(n == (int)strlen(msg)); + for (int i = 0; i < n; i++) { + assert(out[i].c == (uint32_t)msg[i]); + assert(out[i].fg == TERM_COLOR_FG); + assert(out[i].bg == TERM_COLOR_BG); + assert(out[i].flags == 0); + } + + assert(term_hist_decode_segment(&h, 0, 0, 5, out, NULL) == 5); + assert(out[0].c == 'H' && out[4].c == 'o'); + assert(term_hist_decode_segment(&h, 0, 1, 5, out, NULL) == 5); + assert(out[0].c == ',' && out[4].c == 'r'); + assert(term_hist_decode_segment(&h, 0, 2, 5, out, NULL) == 3); + assert(out[0].c == 'l' && out[2].c == '!'); + + term_hist_free(&h); +} + +static void test_attributes_roundtrip(void) +{ + struct hist h; + assert(term_hist_init(&h, 100, 64 * 1024 * 4)); + + struct cell row[10]; + for (int i = 0; i < 10; i++) { + row[i] = (struct cell){ + .c = 'A' + i, + .fg = term_color_rgb(i * 10, i * 20, i * 25), + .bg = term_color_rgb(255 - i * 10, 0, 0), + .flags = (i % 2 == 0) ? CELL_BOLD : CELL_ITALIC, + .pad = (uint16_t)(100 + i), + }; + } + term_hist_push_row(&h, row, 10, false, NULL); + assert(lines(&h) == 1); + + struct cell out[10]; + assert(term_hist_decode_segment(&h, 0, 0, 10, out, NULL) == 10); + for (int i = 0; i < 10; i++) { + assert(out[i].c == row[i].c); + assert(out[i].fg == row[i].fg); + assert(out[i].bg == row[i].bg); + assert(out[i].flags == row[i].flags); + assert(out[i].pad == row[i].pad); + } + term_hist_free(&h); +} + +static void test_wide_and_wrap(void) +{ + struct hist h; + assert(term_hist_init(&h, 100, 64 * 1024 * 4)); + + struct cell row[4] = { dc('A'), dc(0x4e16), dc(' '), dc('B') }; + row[1].flags = CELL_WIDE; + row[2].flags = CELL_TRAIL; + term_hist_push_row(&h, row, 4, false, NULL); + + assert(term_hist_line_rows(&h, 0, 4) == 1); + /* Width 2: 'A' and a pad, then the wide cell, then 'B'. */ + assert(term_hist_line_rows(&h, 0, 2) == 3); + + struct cell out[4]; + assert(term_hist_decode_segment(&h, 0, 0, 2, out, NULL) == 2); + assert(out[0].c == 'A'); + assert(out[1].flags & CELL_WRAP_PAD); + assert(term_hist_decode_segment(&h, 0, 1, 2, out, NULL) == 2); + assert(out[0].c == 0x4e16 && (out[0].flags & CELL_WIDE)); + assert(out[1].flags & CELL_TRAIL); + assert(term_hist_decode_segment(&h, 0, 2, 2, out, NULL) == 1); + assert(out[0].c == 'B'); + term_hist_free(&h); +} + +static void test_clusters_roundtrip(void) +{ + struct gstore *g = term_gstore_new(NULL); + assert(g); + uint32_t cps[3] = { 'e', 0x0301, 0x0300 }; + uint32_t handle = term_gstore_intern(g, cps, 3); + + struct hist h; + assert(term_hist_init(&h, 100, 64 * 1024 * 4)); + struct cell row[2] = { dc(handle), dc('X') }; + term_hist_push_row(&h, row, 2, false, g); + + struct cell out[2]; + assert(term_hist_decode_segment(&h, 0, 0, 2, out, g) == 2); + assert(grapheme_is_handle(out[0].c)); + uint32_t got[GRAPHEME_MAX_LEN]; + assert(term_gstore_expand(g, out[0].c, got, GRAPHEME_MAX_LEN) == 3); + assert(got[0] == cps[0] && got[1] == cps[1] && got[2] == cps[2]); + assert(out[1].c == 'X'); + + term_hist_free(&h); + term_gstore_free(g); +} + +static void test_open_lines_and_pop(void) +{ + struct hist h; + assert(term_hist_init(&h, 100, 64 * 1024 * 4)); + + struct cell part1[3] = { dc('1'), dc('2'), dc('3') }; + term_hist_push_row(&h, part1, 3, true, NULL); + assert(lines(&h) == 1 && hist_is_open(&h)); + struct cell part2[2] = { dc('4'), dc('5') }; + term_hist_push_row(&h, part2, 2, false, NULL); + assert(lines(&h) == 1 && !hist_is_open(&h)); + + struct cell out[5]; + assert(term_hist_decode_segment(&h, 0, 0, 10, out, NULL) == 5); + for (int i = 0; i < 5; i++) + assert(out[i].c == (uint32_t)('1' + i)); + + struct cell popped[2]; + assert(term_hist_pop_tail_cells(&h, 2, popped, NULL) == 2); + assert(popped[0].c == '4' && popped[1].c == '5'); + assert(term_hist_decode_segment(&h, 0, 0, 10, out, NULL) == 3); + assert(out[0].c == '1' && out[2].c == '3'); + + /* Pop from an open line with a pen. */ + term_hist_close_open_line(&h); + struct cell color[2] = { + { .c = 'A', .fg = 0xff0000, .bg = 0x00ff00, .flags = CELL_BOLD }, + { .c = 'B', .fg = 0xff0000, .bg = 0x00ff00, .flags = CELL_BOLD }, + }; + term_hist_push_row(&h, color, 2, true, NULL); + assert(lines(&h) == 2); + struct cell pop_color[1]; + assert(term_hist_pop_tail_cells(&h, 1, pop_color, NULL) == 1); + assert(pop_color[0].c == 'B' && pop_color[0].fg == 0xff0000 && pop_color[0].bg == 0x00ff00); + assert(pop_color[0].flags & CELL_BOLD); + assert(term_hist_decode_segment(&h, 1, 0, 10, out, NULL) == 1); + assert(out[0].c == 'A' && out[0].fg == 0xff0000 && (out[0].flags & CELL_BOLD)); + + /* Popping a whole line returns its cells and removes it. */ + struct cell all[4]; + assert(term_hist_pop_line(&h, all, 4, NULL) == 1); + assert(all[0].c == 'A'); + assert(lines(&h) == 1 && !hist_is_open(&h)); + term_hist_free(&h); +} + +static void test_open_line_joins(void) +{ + struct hist h; + assert(term_hist_init(&h, 100, 64 * 1024 * 4)); + + struct cell line1[2] = { dc('X'), dc('Y') }; + term_hist_push_row(&h, line1, 2, true, NULL); + term_hist_close_open_line(&h); + struct cell line2[2] = { dc('Z'), dc('W') }; + term_hist_push_row(&h, line2, 2, false, NULL); + assert(lines(&h) == 2); + + struct cell out[16]; + assert(term_hist_decode_segment(&h, 0, 0, 10, out, NULL) == 2); + assert(out[0].c == 'X' && out[1].c == 'Y'); + assert(term_hist_decode_segment(&h, 1, 0, 10, out, NULL) == 2); + assert(out[0].c == 'Z' && out[1].c == 'W'); + + /* ASCII joins a wide open line. */ + struct cell wide[2] = { { .c = 0x4e2d, .fg = 0x123456, .bg = TERM_COLOR_BG, .flags = CELL_WIDE }, + { .c = 0, .fg = 0x123456, .bg = TERM_COLOR_BG, .flags = CELL_TRAIL } }; + term_hist_push_row(&h, wide, 2, true, NULL); + term_hist_push_ascii(&h, "hello", 5, NULL, false); + assert(lines(&h) == 3); + assert(term_hist_decode_segment(&h, 2, 0, 10, out, NULL) == 7); + assert(out[0].c == 0x4e2d && out[0].fg == 0x123456); + assert(out[2].c == 'h' && out[2].fg == TERM_COLOR_FG); + term_hist_free(&h); +} + +static void test_eviction(void) +{ + struct hist h; + assert(term_hist_init(&h, 5, 64 * 1024 * 2)); + for (int i = 0; i < 10; i++) { + char buf[16]; + snprintf(buf, sizeof(buf), "line%d", i); + term_hist_push_ascii(&h, buf, (int)strlen(buf), NULL, false); + } + assert(lines(&h) == 5); + assert(hist_first_id(&h) == 5 && hist_end_id(&h) == 10); + + struct cell out[16]; + assert(term_hist_decode_segment(&h, 5, 0, 16, out, NULL) == 5); + assert(out[0].c == 'l' && out[4].c == '5'); + assert(term_hist_decode_segment(&h, 9, 0, 16, out, NULL) == 5); + assert(out[4].c == '9'); + assert(term_hist_decode_segment(&h, 4, 0, 16, out, NULL) == 0); + term_hist_free(&h); +} + +static void test_emoji_and_text_presentation(void) +{ + struct hist h; + assert(term_hist_init(&h, 100, 64 * 1024 * 4)); + struct cell row[3] = { dc(0x2764), dc(' '), dc(0x2600) }; + row[0].flags = CELL_WIDE | CELL_EMOJI; + row[1].flags = CELL_TRAIL; + row[2].flags = CELL_TEXT; + term_hist_push_row(&h, row, 3, false, NULL); + + struct cell out[3]; + assert(term_hist_decode_segment(&h, 0, 0, 10, out, NULL) == 3); + assert(out[0].c == 0x2764 && (out[0].flags & CELL_WIDE) && (out[0].flags & CELL_EMOJI)); + assert(out[1].flags & CELL_TRAIL); + assert(out[2].c == 0x2600 && (out[2].flags & CELL_TEXT)); + term_hist_free(&h); +} + +static void test_width_one(void) +{ + struct hist h; + assert(term_hist_init(&h, 100, 64 * 1024 * 4)); + term_hist_push_ascii(&h, "ABC", 3, NULL, false); + assert(term_hist_line_rows(&h, 0, 1) == 3); + + struct cell out[2]; + assert(term_hist_decode_segment(&h, 0, 0, 1, out, NULL) == 1 && out[0].c == 'A'); + assert(term_hist_decode_segment(&h, 0, 1, 1, out, NULL) == 1 && out[0].c == 'B'); + assert(term_hist_decode_segment(&h, 0, 2, 1, out, NULL) == 1 && out[0].c == 'C'); + + struct cell wide[2] = { dc(0x4e16), dc(' ') }; + wide[0].flags = CELL_WIDE; + wide[1].flags = CELL_TRAIL; + term_hist_push_row(&h, wide, 2, false, NULL); + assert(term_hist_line_rows(&h, 1, 1) == 1); + assert(term_hist_decode_segment(&h, 1, 0, 1, out, NULL) == 1); + assert(out[0].c == 0x4e16 && (out[0].flags & CELL_WIDE)); + term_hist_free(&h); +} + +static void test_blank_lines(void) +{ + struct hist h; + assert(term_hist_init(&h, 10, 64 * 1024 * 4)); + const char *first = "First line on page 0"; + term_hist_push_ascii(&h, first, (int)strlen(first), NULL, false); + term_hist_push_blank(&h, 10); + assert(lines(&h) == 10); + assert(hist_first_id(&h) == 1); + + for (int i = 0; i < 20; i++) { + char buf[32]; + snprintf(buf, sizeof(buf), "line %d", i); + term_hist_push_ascii(&h, buf, (int)strlen(buf), NULL, false); + } + struct cell out[32]; + assert(term_hist_decode_segment(&h, hist_end_id(&h) - 1, 0, 32, out, NULL) == 7); + assert(out[5].c == '1' && out[6].c == '9'); + + /* A blank row closes an open line and adds no line. */ + term_hist_push_ascii(&h, "open", 4, NULL, true); + uint64_t before = hist_end_id(&h); + term_hist_push_blank(&h, 1); + assert(!hist_is_open(&h) && hist_end_id(&h) == before); + + /* More blank rows than the limit keep the id count going. */ + term_hist_push_blank(&h, 1000); + assert(lines(&h) == 10 && hist_end_id(&h) == before + 1000); + term_hist_push_blank(&h, 5); + assert(lines(&h) == 10 && hist_end_id(&h) == before + 1005); + assert(term_hist_decode_segment(&h, hist_end_id(&h) - 1, 0, 32, out, NULL) == 0); + assert(term_hist_line_rows(&h, hist_end_id(&h) - 1, 32) == 1); + term_hist_free(&h); +} + +static void test_open_line_attrs_continuation(void) +{ + struct hist h; + assert(term_hist_init(&h, 100, 64 * 1024 * 4)); + struct cell r1[5], r2[10]; + for (int i = 0; i < 5; i++) + r1[i] = (struct cell){ .c = '1' + i, .fg = term_color_rgb(255, 0, 0), .bg = TERM_COLOR_BG }; + for (int i = 0; i < 10; i++) + r2[i] = (struct cell){ .c = 'a' + i, .fg = i < 5 ? term_color_rgb(255, 0, 0) : term_color_rgb(0, 0, 255), + .bg = TERM_COLOR_BG }; + term_hist_push_row(&h, r1, 5, true, NULL); + term_hist_push_row(&h, r2, 10, false, NULL); + assert(lines(&h) == 1); + + struct cell out[15]; + assert(term_hist_decode_segment(&h, 0, 0, 15, out, NULL) == 15); + for (int i = 0; i < 10; i++) + assert(out[i].fg == term_color_rgb(255, 0, 0)); + for (int i = 10; i < 15; i++) + assert(out[i].fg == term_color_rgb(0, 0, 255)); + term_hist_free(&h); +} + +static void mark_link(void *ud, uint16_t link) +{ + ((int *)ud)[link]++; +} + +static void test_hyperlinks(void) +{ + struct hist h; + assert(term_hist_init(&h, 100, 64 * 1024 * 4)); + struct pen_attr pen = pen_attr_default(); + pen.link = 42; + term_hist_push_ascii(&h, "Link text", 9, &pen, false); + struct cell row[2] = { dc('a'), dc('b') }; + row[1].pad = 7; + row[1].flags = CELL_LINK; + term_hist_push_row(&h, row, 2, true, NULL); + + struct cell out[16]; + assert(term_hist_decode_segment(&h, 0, 0, 16, out, NULL) == 9); + for (int i = 0; i < 9; i++) + assert(out[i].pad == 42); + + int marks[64] = { 0 }; + term_hist_mark_links(&h, mark_link, marks); + assert(marks[42] && marks[7]); + term_hist_free(&h); +} + +static void test_long_lines(void) +{ + struct hist h; + assert(term_hist_init(&h, 100, 64 * 1024 * 4)); + + /* A line past the part limit keeps one id. */ + char chunk[1024]; + memset(chunk, 'X', sizeof(chunk)); + for (int i = 0; i < 64; i++) + term_hist_push_ascii(&h, chunk, sizeof(chunk), NULL, true); + term_hist_push_ascii(&h, chunk, 100, NULL, false); + assert(lines(&h) == 1); + assert(term_hist_line_cells(&h, 0) == 65636); + assert(term_hist_line_rows(&h, 0, 1000) == 66); + + struct cell out[1000]; + assert(term_hist_decode_segment(&h, 0, 64, 1000, out, NULL) == 1000); + for (int i = 0; i < 1000; i++) + assert(out[i].c == 'X'); + assert(term_hist_decode_segment(&h, 0, 65, 1000, out, NULL) == 636); + + memset(chunk, 'Y', sizeof(chunk)); + for (int i = 0; i < 64; i++) + term_hist_push_ascii(&h, chunk, sizeof(chunk), NULL, true); + term_hist_push_ascii(&h, chunk, 100, NULL, false); + term_hist_push_ascii(&h, "tail", 4, NULL, false); + assert(lines(&h) == 3); + assert(term_hist_line_rows(&h, 1, 1000) == 66); + assert(term_hist_decode_segment(&h, 1, 0, 1000, out, NULL) == 1000 && out[0].c == 'Y'); + assert(term_hist_decode_segment(&h, 2, 0, 1000, out, NULL) == 4 && out[0].c == 't'); + assert(term_hist_rows(&h, 1000) == 133); + + /* One long ASCII push splits into parts but reads back whole. */ + term_hist_free(&h); + assert(term_hist_init(&h, 100, 64 * 1024 * 8)); + int big_len = 150000; + char *big = malloc((size_t)big_len); + assert(big); + for (int i = 0; i < big_len; i++) + big[i] = (char)('A' + (i % 26)); + term_hist_push_ascii(&h, big, big_len, NULL, false); + assert(lines(&h) == 1); + assert(term_hist_line_rows(&h, 0, 1000) == 150); + for (int s = 0; s < 150; s++) { + assert(term_hist_decode_segment(&h, 0, s, 1000, out, NULL) == 1000); + for (int x = 0; x < 1000; x++) + assert(out[x].c == (uint32_t)big[s * 1000 + x]); + } + free(big); + term_hist_free(&h); +} + +static void test_page_limits(void) +{ + struct hist h; + struct cell out[1000]; + + /* One page: old lines leave to make room. */ + assert(term_hist_init(&h, 10000, HIST_PAGE_SIZE)); + char line64[64]; + memset(line64, 'M', sizeof(line64)); + for (int i = 0; i < 1200; i++) + term_hist_push_ascii(&h, line64, sizeof(line64), NULL, false); + assert(lines(&h) > 0 && lines(&h) < 1200); + assert(term_hist_decode_segment(&h, hist_end_id(&h) - 1, 0, 64, out, NULL) == 64); + assert(out[0].c == 'M'); + term_hist_free(&h); + + /* A line limit of one keeps the long line as one line. */ + assert(term_hist_init(&h, 1, 64 * 1024 * 4)); + char chunk[1024]; + memset(chunk, 'Z', sizeof(chunk)); + for (int i = 0; i < 65; i++) + term_hist_push_ascii(&h, chunk, sizeof(chunk), NULL, true); + assert(lines(&h) == 1 && hist_is_open(&h)); + term_hist_push_ascii(&h, "new", 3, NULL, false); + term_hist_push_ascii(&h, "next", 4, NULL, false); + assert(lines(&h) == 1); + assert(term_hist_decode_segment(&h, hist_first_id(&h), 0, 10, out, NULL) == 4); + term_hist_free(&h); + + /* A line longer than the arena drops its head but keeps its id. */ + assert(term_hist_init(&h, 10, HIST_PAGE_SIZE)); + term_hist_push_ascii(&h, "PREFIX", 6, NULL, true); + for (int i = 0; i < 100; i++) + term_hist_push_ascii(&h, chunk, sizeof(chunk), NULL, true); + term_hist_close_open_line(&h); + assert(lines(&h) == 1 && hist_first_id(&h) == 0); + assert(term_hist_line_cells(&h, 0) > 0 && term_hist_line_cells(&h, 0) < 100 * 1024); + assert(term_hist_decode_segment(&h, 0, 0, 6, out, NULL) == 6 && out[0].c == 'Z'); + term_hist_free(&h); + + /* A small open line moves to a new page instead of splitting. */ + assert(term_hist_init(&h, 10, HIST_PAGE_SIZE)); + for (int i = 0; i < 60; i++) + term_hist_push_ascii(&h, chunk, sizeof(chunk), NULL, false); + term_hist_push_ascii(&h, "PREFIX", 6, NULL, true); + for (int i = 0; i < 10; i++) + term_hist_push_ascii(&h, chunk, sizeof(chunk), NULL, true); + term_hist_close_open_line(&h); + uint64_t id = hist_end_id(&h) - 1; + assert(term_hist_decode_segment(&h, id, 0, 6, out, NULL) == 6); + assert(out[0].c == 'P' && out[5].c == 'X'); + assert(term_hist_line_rows(&h, id, 1000) == 11); + term_hist_free(&h); +} + +static void test_nul_and_wide_pop(void) +{ + struct hist h; + struct cell out[8]; + assert(term_hist_init(&h, 10, 64 * 1024)); + struct cell nul_row[4] = { dc('A'), dc(0), dc('B'), dc('C') }; + term_hist_push_row(&h, nul_row, 4, false, NULL); + assert(term_hist_decode_segment(&h, 0, 0, 4, out, NULL) == 4); + assert(out[0].c == 'A' && out[1].c == ' ' && out[2].c == 'B' && out[3].c == 'C'); + term_hist_free(&h); + + /* A pop does not split a wide cell. */ + assert(term_hist_init(&h, 10, 64 * 1024 * 4)); + struct cell wide[4] = { dc('A'), dc(0x4E2D), dc(0), dc('B') }; + wide[1].flags = CELL_WIDE; + wide[2].flags = CELL_TRAIL; + term_hist_push_row(&h, wide, 4, true, NULL); + struct cell popped[2]; + assert(term_hist_pop_tail_cells(&h, 2, popped, NULL) == 1); + assert(popped[0].c == 'B'); + assert(term_hist_decode_segment(&h, 0, 0, 4, out, NULL) == 3); + assert(out[0].c == 'A' && out[1].c == 0x4E2D && (out[2].flags & CELL_TRAIL)); + term_hist_free(&h); +} + +static void test_default_prefix_attributes(void) +{ + struct hist h; + assert(term_hist_init(&h, 100, 64 * 1024 * 4)); + struct cell row[8]; + for (int i = 0; i < 8; i++) { + row[i] = dc('a' + i); + if (i >= 4) { + row[i].fg = term_color_rgb(10, 20, 30); + row[i].bg = term_color_rgb(40, 50, 60); + row[i].flags = CELL_BOLD; + } + } + term_hist_push_row(&h, row, 8, false, NULL); + struct cell out[8]; + assert(term_hist_decode_segment(&h, 0, 0, 8, out, NULL) == 8); + for (int i = 0; i < 8; i++) { + assert(out[i].c == (uint32_t)('a' + i)); + assert(out[i].fg == row[i].fg && out[i].bg == row[i].bg && out[i].flags == row[i].flags); + } + + struct pen_attr colored = pen_attr_default(); + colored.fg = term_color_rgb(200, 100, 50); + term_hist_push_ascii(&h, "0123", 4, NULL, true); + term_hist_push_ascii(&h, "4567", 4, &colored, false); + assert(term_hist_decode_segment(&h, 1, 0, 8, out, NULL) == 8); + for (int i = 0; i < 8; i++) + assert(out[i].fg == (i < 4 ? TERM_COLOR_FG : term_color_rgb(200, 100, 50))); + term_hist_free(&h); +} + +static void test_utf8_and_disabled(void) +{ + struct hist h; + assert(term_hist_init(&h, 2, 64 * 1024 * 2)); + term_hist_push_ascii(&h, "first", 5, NULL, false); + struct cell p1[2] = { dc('A'), dc('B') }; + struct cell p2[2] = { dc(0x00e9), dc('C') }; + term_hist_push_row(&h, p1, 2, true, NULL); + term_hist_push_row(&h, p2, 2, false, NULL); + assert(lines(&h) == 2 && hist_first_id(&h) == 0); + struct cell out[8]; + assert(term_hist_decode_segment(&h, 1, 0, 8, out, NULL) == 4); + assert(out[0].c == 'A' && out[1].c == 'B' && out[2].c == 0x00e9 && out[3].c == 'C'); + term_hist_free(&h); + + assert(term_hist_init(&h, 0, 0)); + term_hist_push_ascii(&h, "ignored", 7, NULL, false); + term_hist_push_row(&h, p1, 2, false, NULL); + term_hist_push_blank(&h, 3); + assert(lines(&h) == 0); + term_hist_free(&h); +} + +/* Random rows of every kind, pushed and compared with a plain copy. */ +static void test_fuzz_rows(void) +{ + enum { LIMIT = 40, COLS = 24, ITER = 4000 }; + struct gstore *g = term_gstore_new(NULL); + uint32_t cps[2] = { 'a', 0x0301 }; + uint32_t cluster = term_gstore_intern(g, cps, 2); + struct hist h; + assert(term_hist_init(&h, LIMIT, 64 * 1024 * 2)); + + static struct cell ref[LIMIT + 1][COLS * 8]; + static int ref_len[LIMIT + 1]; + uint64_t ref_first = 0, ref_end = 0; + bool open = false; + unsigned state = 7; + for (int it = 0; it < ITER; it++) { + state = state * 1103515245u + 12345u; + int len = (int)((state >> 16) % (COLS + 1)); + bool wrapped = ((state >> 8) & 3) == 0; + struct cell row[COLS]; + int n = 0; + while (n < len) { + state = state * 1103515245u + 12345u; + unsigned k = (state >> 16) % 10; + struct cell c = dc('a' + (state >> 20) % 26); + if (k == 1) + c.fg = term_color_index((int)(state >> 24) % 16); + if (k == 2) + c.flags = CELL_BOLD; + if (k == 3) + c.c = 0xe9; + if (k == 4) + c.c = cluster; + if (k == 5 && n + 1 < len) { + c.c = 0x4e00 + (state >> 20) % 100; + c.flags = CELL_WIDE; + row[n++] = c; + c.c = 0; + c.flags = CELL_TRAIL; + } + row[n++] = c; + } + if (!open) { + if (ref_end - ref_first == LIMIT) + ref_first++; + ref_len[ref_end % (LIMIT + 1)] = 0; + ref_end++; + } + int slot = (int)((ref_end - 1) % (LIMIT + 1)); + if (!open && len == 0 && !wrapped) { + /* A blank line. */ + } else { + memcpy(&ref[slot][ref_len[slot]], row, (size_t)n * sizeof(row[0])); + ref_len[slot] += n; + } + term_hist_push_row(&h, row, n, wrapped, g); + open = wrapped; + assert(hist_first_id(&h) == ref_first && hist_end_id(&h) == ref_end); + } + + struct cell out[COLS * 8]; + for (uint64_t id = ref_first; id < ref_end; id++) { + int slot = (int)(id % (LIMIT + 1)); + int want = ref_len[slot]; + assert((int)term_hist_line_cells(&h, id) == want); + if (!want) + continue; + assert(term_hist_decode_segment(&h, id, 0, COLS * 8, out, g) == want); + for (int x = 0; x < want; x++) { + const struct cell *a = &ref[slot][x], *b = &out[x]; + if (a->flags & CELL_TRAIL) { + assert(b->flags & CELL_TRAIL); + continue; + } + assert(a->fg == b->fg && a->flags == b->flags); + if (grapheme_is_handle(a->c)) { + uint32_t got[GRAPHEME_MAX_LEN]; + assert(term_gstore_expand(g, b->c, got, GRAPHEME_MAX_LEN) == 2 && got[1] == 0x0301); + } else { + assert(a->c == b->c); + } + } + } + term_hist_free(&h); + term_gstore_free(g); +} + +static void test_large_wide_count(void) +{ + struct hist h; + struct hist_view view; + assert(term_hist_init(&h, 600, 4 * HIST_PAGE_SIZE)); + term_hist_view_init(&view); + struct cell row[4] = { dc('A'), dc(0x65e5), dc(0), dc('B') }; + row[1].flags = CELL_WIDE; + row[2].flags = CELL_TRAIL; + for (int i = 0; i < 400; i++) + term_hist_push_row(&h, row, 4, true, NULL); + term_hist_close_open_line(&h); + uint64_t exact = term_hist_rows(&h, 3); + term_hist_prepare_rows(&h, 3); + assert(term_hist_estimated_rows(&h, 3) < exact); + assert(term_hist_view_sync(&view, &h, 3)); + term_hist_view_scroll_to(&view, &h, 3, (struct line_pos){0}, INT_MAX); + term_hist_view_refine(&view, &h, 3, 3); + assert(exact > 254); + assert(term_hist_estimated_rows(&h, 3) == exact); + for (int i = 0; i < 300; i++) + term_hist_push_ascii(&h, "x", 1, NULL, false); + assert(term_hist_estimated_rows(&h, 3) == exact + 300); + assert(term_hist_view_sync(&view, &h, 3)); + for (int i = 0; i < 300; i++) + term_hist_push_ascii(&h, "x", 1, NULL, false); + assert(term_hist_estimated_rows(&h, 3) == 600); + term_hist_view_free(&view); + term_hist_free(&h); +} + +static void test_view_after_prefix_eviction(void) +{ + struct hist h; + struct hist_view view; + assert(term_hist_init(&h, 10, 2 * HIST_PAGE_SIZE)); + term_hist_view_init(&view); + char bytes[1000]; + for (int i = 0; i < 100; i++) { + memset(bytes, 'A' + i % 26, sizeof(bytes)); + term_hist_push_ascii(&h, bytes, sizeof(bytes), NULL, true); + } + term_hist_view_scroll_to(&view, &h, 1000, (struct line_pos){0}, 10); + uint64_t id; + int seg, screen_y; + struct cell before[1000], after[1000]; + assert(term_hist_view_row_pos(&view, &h, 1000, 3, (struct line_pos){0}, + 0, &id, &seg, &screen_y)); + assert(term_hist_decode_segment(&h, id, seg, 1000, before, NULL) == 1000); + for (int i = 100; i < 140; i++) { + memset(bytes, 'A' + i % 26, sizeof(bytes)); + term_hist_push_ascii(&h, bytes, sizeof(bytes), NULL, true); + term_hist_view_note_push(&view, &h, 1000); + } + assert(term_hist_line_cells(&h, 0) < 140000); + assert(term_hist_view_row_pos(&view, &h, 1000, 3, (struct line_pos){0}, + 0, &id, &seg, &screen_y)); + assert(term_hist_decode_segment(&h, id, seg, 1000, after, NULL) == 1000); + for (int i = 0; i < 1000; i++) + assert(before[i].c == after[i].c); + term_hist_view_rewrap(&view, &h, 500); + assert(term_hist_view_row_pos(&view, &h, 500, 3, (struct line_pos){0}, + 0, &id, &seg, &screen_y)); + assert(term_hist_decode_segment(&h, id, seg, 500, after, NULL) == 500); + for (int i = 0; i < 500; i++) + assert(before[i].c == after[i].c); + term_hist_view_free(&view); + term_hist_free(&h); +} + +static void test_short_lines_after_eviction(void) +{ + struct hist h; + assert(term_hist_init(&h, 64, 64 * 1024 * 4)); + term_hist_prepare_rows(&h, 80); + const char text[] = "abc\n\nx\npqrs\n\nz\n"; + const int offsets[] = {0, 4, 5, 7, 12, 13}; + const int lengths[] = {3, 0, 1, 4, 0, 1}; + struct hist_ascii_line batch[6]; + for (int i = 0; i < 6; i++) + batch[i] = (struct hist_ascii_line){ + .ptr = text + offsets[i], .len = (uint16_t)lengths[i], + }; + for (int pass = 0; pass < 20000; pass++) + term_hist_push_ascii_lines(&h, batch, 6, NULL); + assert(lines(&h) == 64); + assert(term_hist_rows(&h, 80) == 64); + for (uint64_t id = hist_first_id(&h); id < hist_end_id(&h); id++) { + int k = (int)(id % 6); + struct cell out[80]; + assert(term_hist_decode_segment(&h, id, 0, 80, out, NULL) == lengths[k]); + for (int x = 0; x < lengths[k]; x++) { + assert(out[x].c == (uint32_t)text[offsets[k] + x]); + assert(out[x].fg == TERM_COLOR_FG); + assert(out[x].bg == TERM_COLOR_BG); + } + } + term_hist_free(&h); +} + +int main(void) +{ + test_short_lines_after_eviction(); + test_large_wide_count(); + test_view_after_prefix_eviction(); + test_init_free(); + test_ascii_roundtrip(); + test_attributes_roundtrip(); + test_wide_and_wrap(); + test_clusters_roundtrip(); + test_open_lines_and_pop(); + test_open_line_joins(); + test_eviction(); + test_width_one(); + test_emoji_and_text_presentation(); + test_blank_lines(); + test_open_line_attrs_continuation(); + test_hyperlinks(); + test_long_lines(); + test_page_limits(); + test_nul_and_wide_pop(); + test_default_prefix_attributes(); + test_utf8_and_disabled(); + test_fuzz_rows(); + + printf("PASS: hist tests\n"); + return 0; +} diff --git a/tests/history_text_test.c b/tests/history_text_test.c new file mode 100644 index 0000000..88ca6aa --- /dev/null +++ b/tests/history_text_test.c @@ -0,0 +1,135 @@ +#include "term/term.h" +#include "term_api.h" + +#include <assert.h> +#include <pthread.h> +#include <stdatomic.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> + +static struct term *new_term(int cols) +{ + struct term *term = term_new(&(struct term_options){ + .size = { .x = cols, .y = 2 }, + .scrollback = 100, + }); + assert(term); + return term; +} + +static void feed(struct term *term, const char *text) +{ + term_feed(term, text, strlen(text), 0); +} + +static void expect_history(struct term *term, int rows, const char *expected) +{ + char *text = tt_history_text(term, rows); + assert(text); + if (strcmp(text, expected)) { + fprintf(stderr, "history: got \"%s\" want \"%s\"\n", text, expected); + assert(0); + } + free(text); +} + +static void unicode_and_selection(void) +{ + struct term *term = new_term(16); + expect_history(term, 100, ""); + feed(term, "é👩💻界\r\nnext\r\nlast"); + expect_history(term, 100, "é👩💻界\n"); + expect_history(term, 0, ""); + term_select_start(term, (struct term_vector2i){0}, false, TERM_SELECT_CELL); + term_select_extend(term, (struct term_vector2i){ .x = 4 }, false); + char *before = tt_selection_text(term); + assert(before); + expect_history(term, 100, "é👩💻界\n"); + char *after = tt_selection_text(term); + assert(after && !strcmp(before, after)); + free(before); + free(after); + feed(term, "\033[?1049h"); + expect_history(term, 100, ""); + feed(term, "\033[?1049l"); + expect_history(term, 100, "é👩💻界\n"); + term_free(term); +} + +static void limits_and_scrolled_view(void) +{ + struct term *term = new_term(8); + feed(term, "one\r\ntwo\r\nthree\r\nfour\r\nfive"); + expect_history(term, 100, "one\ntwo\nthree\n"); + expect_history(term, 2, "two\nthree\n"); + term_scroll(term, 2); + expect_history(term, 100, "one\n"); + term_scroll(term, 100); + expect_history(term, 100, ""); + term_scroll_to(term, 0); + assert(term_resize(term, (struct term_vector2i){ .x = 12, .y = 2 })); + expect_history(term, 100, "one\ntwo\nthree\n"); + term_free(term); +} + +static void wrapped_rows(void) +{ + struct term *term = new_term(4); + feed(term, "abcdefghi\r\nx\r\ny"); + expect_history(term, 100, "abcdefghi\n"); + expect_history(term, 2, "efghi\n"); + term_free(term); + + term = new_term(4); + feed(term, "abcdefghijklm"); + expect_history(term, 100, "abcdefgh"); + term_free(term); +} + +struct writer { + struct term *term; + _Atomic bool done; +}; + +static const char unicode_line[] = "é👩💻界\r\n"; +static const char history_line[] = "é👩💻界\n"; + +static void *write_history(void *userdata) +{ + struct writer *writer = userdata; + for (int i = 0; i < 1000; i++) + feed(writer->term, unicode_line); + atomic_store(&writer->done, true); + return NULL; +} + +static void concurrent_history(void) +{ + struct term *term = new_term(16); + struct writer writer = { .term = term }; + pthread_t thread; + assert(pthread_create(&thread, NULL, write_history, &writer) == 0); + do { + char *text = tt_history_text(term, 10); + assert(text); + size_t n = strlen(text); + size_t line_len = sizeof(history_line) - 1; + assert(n % line_len == 0 && n / line_len <= 10); + for (size_t offset = 0; offset < n; offset += line_len) + assert(!memcmp(text + offset, history_line, line_len)); + free(text); + } while (!atomic_load(&writer.done)); + assert(pthread_join(thread, NULL) == 0); + term_free(term); +} + +int main(void) +{ + unicode_and_selection(); + limits_and_scrolled_view(); + wrapped_rows(); + concurrent_history(); + puts("PASS: history text"); + return 0; +} diff --git a/tests/input_test.c b/tests/input_test.c new file mode 100644 index 0000000..6ee2caa --- /dev/null +++ b/tests/input_test.c @@ -0,0 +1,353 @@ +#include "term/term.h" +#include <assert.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> + +static struct term_keyboard_state st; +static struct term_modes m; +static char out[128]; +static size_t out_len; + +static void mode(unsigned kitty_flags) +{ + st = (struct term_keyboard_state){0}; + m = (struct term_modes){ .keyboard = kitty_flags, .grid = { .x = 80, .y = 24 } }; +} + +static void enc(struct term_key_event ev) +{ + if (!ev.action) ev.action = TERM_KEY_PRESS; + term_keyboard_update(&st, &ev); + struct term_input_result r = + term_input_encode_key(&st, &ev, &m, out, sizeof(out)); + out_len = r.len; +} +#define K(...) enc((struct term_key_event){ __VA_ARGS__ }) + +static void typed(uint32_t cp, unsigned mods) +{ + K(.codepoint = cp, .base = cp, .text = cp, .mods = mods); +} + +static bool consumed; +static void enc_mouse(struct term_mouse_event ev) +{ + if (ev.pixel.x == 0 && ev.pixel.y == 0) + ev.pixel = ev.pos; + struct term_input_result r = + term_input_encode_mouse(&ev, &m, out, sizeof(out)); + out_len = r.len; + consumed = r.status != TERM_INPUT_UNHANDLED; +} +#define MOUSE(x_, y_, ...) enc_mouse((struct term_mouse_event){ .pos = { .x = (x_), .y = (y_) }, __VA_ARGS__ }) + +static void expect_at(const char *s, int line) +{ + if (out_len != strlen(s) || memcmp(out, s, out_len)) { + fprintf(stderr, "line %d: expected %s, got %.*s (%zu bytes)\n", line, s, + (int)out_len, out, out_len); + abort(); + } +} +#define expect(s) expect_at(s, __LINE__) + +int main(void) +{ + mode(0); + K(.key = TERM_KEY_UP); expect("\033[A"); + K(.key = TERM_KEY_HOME); expect("\033[H"); + K(.key = TERM_KEY_DELETE); expect("\033[3~"); + K(.key = TERM_KEY_PAGE_UP); expect("\033[5~"); + K(.key = TERM_KEY_F1); expect("\033OP"); + K(.key = TERM_KEY_F5); expect("\033[15~"); + K(.key = TERM_KEY_F12); expect("\033[24~"); + K(.key = TERM_KEY_RETURN); expect("\r"); + K(.key = TERM_KEY_ESCAPE); expect("\033"); + K(.key = TERM_KEY_TAB); expect("\t"); + K(.key = TERM_KEY_TAB, .mods = TERM_MOD_SHIFT); expect("\033[Z"); + K(.key = TERM_KEY_BACKSPACE); expect("\177"); + K(.key = TERM_KEY_RIGHT, .mods = TERM_MOD_CTRL); expect("\033[1;5C"); + K(.key = TERM_KEY_F5, .mods = TERM_MOD_SHIFT); expect("\033[15;2~"); + K(.key = TERM_KEY_DELETE, .mods = TERM_MOD_CTRL); expect("\033[3;5~"); + + m.backarrow_bs = true; + K(.key = TERM_KEY_BACKSPACE); expect("\010"); + m.backarrow_bs = false; + + m.app_cursor = true; + K(.key = TERM_KEY_UP); expect("\033OA"); + K(.key = TERM_KEY_END); expect("\033OF"); + + K(.key = TERM_KEY_UP, .mods = TERM_MOD_CTRL); expect("\033[1;5A"); + m.app_cursor = false; + + typed('a', 0); expect("a"); + typed(0x00E9, 0); expect("\xc3\xa9"); + K(.codepoint = 'a', .shifted = 'A', .base = 'a', .text = 'A', + .mods = TERM_MOD_SHIFT); expect("A"); + K(.action = TERM_KEY_RELEASE, .codepoint = 'a', .text = 'a'); expect(""); + + typed('c', TERM_MOD_CTRL); expect("\003"); + typed('a', TERM_MOD_CTRL); expect("\001"); + typed('\\', TERM_MOD_CTRL); expect("\034"); + typed(' ', TERM_MOD_CTRL); assert(out_len == 1 && out[0] == 0); + + K(.codepoint = '2', .shifted = '@', .base = '2', .text = '@', + .mods = TERM_MOD_CTRL | TERM_MOD_SHIFT); + assert(out_len == 1 && out[0] == 0); + + typed('b', TERM_MOD_ALT); expect("\033b"); + typed('b', TERM_MOD_ALT | TERM_MOD_CTRL); expect("\033\002"); + K(.key = TERM_KEY_BACKSPACE, .mods = TERM_MOD_ALT); expect("\033\177"); + K(.key = TERM_KEY_RETURN, .mods = TERM_MOD_ALT); expect("\033\r"); + + K(.key = TERM_KEY_LEFT, .mods = TERM_MOD_ALT); expect("\033[1;3D"); + + K(.codepoint = 'b', .base = 'b', .text = 0x222B, .text_deferred = true); + expect(""); + + K(.key = TERM_KEY_KP_DELETE, .keypad = TERM_KEY_KP_DECIMAL); expect("\033[3~"); + K(.key = TERM_KEY_KP_ENTER, .keypad = TERM_KEY_KP_ENTER); expect("\r"); + K(.key = TERM_KEY_KP_LEFT, .keypad = TERM_KEY_KP_4); expect("\033[D"); + + m.app_keypad = true; + K(.key = TERM_KEY_KP_END, .keypad = TERM_KEY_KP_1); expect("\033Oq"); + K(.key = TERM_KEY_KP_ENTER, .keypad = TERM_KEY_KP_ENTER); expect("\033OM"); + K(.key = TERM_KEY_KP_ADD, .keypad = TERM_KEY_KP_ADD); expect("\033Ok"); + m.app_keypad = false; + + mode(1); + K(.key = TERM_KEY_ESCAPE); expect("\033[27u"); + K(.key = TERM_KEY_DELETE); expect("\033[3~"); + K(.key = TERM_KEY_F3); expect("\033[13~"); + K(.key = TERM_KEY_KP_ENTER, .keypad = TERM_KEY_KP_ENTER); expect("\033[57414u"); + typed('i', TERM_MOD_CTRL); expect("\033[105;5u"); + K(.key = TERM_KEY_RETURN, .mods = TERM_MOD_CTRL); expect("\033[13;5u"); + + typed('a', 0); expect("a"); + K(.key = TERM_KEY_RETURN); expect("\r"); + + K(.key = TERM_KEY_UP, .action = TERM_KEY_RELEASE); expect(""); + K(.key = TERM_KEY_LSHIFT); expect(""); + + mode(1 | 4); + K(.codepoint = 'a', .shifted = 'A', .base = 'a', .text = 'A', + .mods = TERM_MOD_CTRL | TERM_MOD_SHIFT); expect("\033[97:65;6u"); + + mode(1 | 2 | 8); + K(.key = TERM_KEY_ESCAPE); expect("\033[27;1u"); + K(.key = TERM_KEY_ESCAPE, .action = TERM_KEY_REPEAT); expect("\033[27;1:2u"); + K(.key = TERM_KEY_ESCAPE, .action = TERM_KEY_RELEASE); expect("\033[27;1:3u"); + K(.key = TERM_KEY_UP, .action = TERM_KEY_REPEAT); expect("\033[1;1:2A"); + + mode(31); + K(.codepoint = 1092, .base = 'a', .text = 1092); + expect("\033[1092::97;1;1092u"); + K(.codepoint = 'a', .shifted = 'A', .base = 'a', .text = 'A', + .mods = TERM_MOD_SHIFT); expect("\033[97:65;2;65u"); + + K(.action = TERM_KEY_RELEASE, .codepoint = 'a', .base = 'a', .text = 'a'); + expect("\033[97;1:3u"); + + mode(1 | 2 | 8); + K(.key = TERM_KEY_LSHIFT, .mods = TERM_MOD_SHIFT); expect("\033[57441;2u"); + K(.key = TERM_KEY_RSHIFT, .mods = TERM_MOD_SHIFT); expect("\033[57447;2u"); + K(.key = TERM_KEY_LSHIFT, .action = TERM_KEY_RELEASE, .mods = TERM_MOD_SHIFT); + expect("\033[57441;2:3u"); + K(.key = TERM_KEY_RSHIFT, .action = TERM_KEY_RELEASE, .mods = TERM_MOD_SHIFT); + expect("\033[57447;1:3u"); + + K(.key = TERM_KEY_ISO_LEVEL3_SHIFT); expect("\033[57453;1u"); + K(.key = TERM_KEY_ISO_LEVEL5_SHIFT); expect("\033[57454;1u"); + K(.key = TERM_KEY_CAPS_LOCK); expect("\033[57358;65u"); + + mode(1 | 2 | 4); + K(.key = TERM_KEY_CAPS_LOCK); expect(""); + K(.key = TERM_KEY_CAPS_LOCK, .action = TERM_KEY_RELEASE); expect(""); + + mode(1 | 2 | 4 | 8); + K(.key = TERM_KEY_CAPS_LOCK); expect("\033[57358;65u"); + K(.key = TERM_KEY_CAPS_LOCK, .action = TERM_KEY_RELEASE, + .mods = TERM_MOD_CAPS); expect("\033[57358;65:3u"); + + mode(1); + K(.key = TERM_KEY_LEFT, .mods = TERM_MOD_NUM); expect("\033[1;129D"); + K(.key = TERM_KEY_LEFT); expect("\033[D"); + K(.key = TERM_KEY_F1, .mods = TERM_MOD_CAPS); expect("\033[1;65P"); + K(.codepoint = 'a', .base = 'a', .mods = TERM_MOD_CTRL | TERM_MOD_CAPS); + expect("\033[97;69u"); + K(.codepoint = 'a', .base = 'a', .mods = TERM_MOD_CTRL); + expect("\033[97;5u"); + K(.codepoint = 'a', .base = 'a', .text = 'A', .mods = TERM_MOD_CAPS); + expect("A"); + K(.key = TERM_KEY_RETURN, .mods = TERM_MOD_CAPS); expect("\r"); + + mode(1 | 8); + K(.codepoint = 'a', .base = 'a', .text = 'A', .mods = TERM_MOD_CAPS); + expect("\033[97;65u"); + K(.key = TERM_KEY_RETURN, .mods = TERM_MOD_CAPS); expect("\033[13;65u"); + + mode(0); + K(.key = TERM_KEY_LEFT, .mods = TERM_MOD_NUM); expect("\033[D"); + K(.key = TERM_KEY_F1, .mods = TERM_MOD_CAPS); expect("\033OP"); + K(.codepoint = 'a', .base = 'a', .mods = TERM_MOD_CTRL | TERM_MOD_CAPS); + expect("\001"); + + mode(0); + MOUSE(2, 1, .kind = TERM_MOUSE_PRESS); + assert(out_len == 0 && !consumed); + + m.mouse = TERM_MOUSE_NORMAL; + MOUSE(2, 1, .kind = TERM_MOUSE_PRESS); expect("\033[M\040\043\042"); + MOUSE(2, 1, .kind = TERM_MOUSE_RELEASE); expect("\033[M\043\043\042"); + MOUSE(2, 1, .kind = TERM_MOUSE_MOTION); expect(""); + + MOUSE(2, 1, .mods = TERM_MOD_SHIFT); + assert(out_len == 0 && !consumed); + + MOUSE(200, 99, .kind = TERM_MOUSE_PRESS); + assert(out_len == 0 && consumed); + + m.mouse_encoding = TERM_MOUSE_ENC_SGR; + MOUSE(2, 1, .kind = TERM_MOUSE_PRESS); expect("\033[<0;3;2M"); + MOUSE(2, 1, .kind = TERM_MOUSE_RELEASE); expect("\033[<0;3;2m"); + MOUSE(2, 1, .mods = TERM_MOD_CTRL); expect("\033[<16;3;2M"); + MOUSE(2, 1, .button = TERM_MOUSE_WHEEL_UP); expect("\033[<64;3;2M"); + m.mouse_encoding = TERM_MOUSE_ENC_SGR_PIXELS; + MOUSE(2, 1, .pixel = { .x = 25, .y = 17 }); expect("\033[<0;26;18M"); + m.mouse_encoding = TERM_MOUSE_ENC_URXVT; + MOUSE(2, 1, .kind = TERM_MOUSE_PRESS); expect("\033[32;3;2M"); + m.mouse_encoding = TERM_MOUSE_ENC_X10; + + m.mouse = TERM_MOUSE_BUTTON; + MOUSE(2, 1, .kind = TERM_MOUSE_MOTION, .button = 3); expect(""); + MOUSE(2, 1, .kind = TERM_MOUSE_MOTION, .button = 0); + expect("\033[M\100\043\042"); + m.mouse_encoding = TERM_MOUSE_ENC_SGR; + MOUSE(2, 1, .kind = TERM_MOUSE_MOTION, .button = 0, .mods = TERM_MOD_CTRL); + expect("\033[<48;3;2M"); + m.mouse = TERM_MOUSE_ANY; + MOUSE(2, 1, .kind = TERM_MOUSE_MOTION, .button = 3); expect("\033[<35;3;2M"); + m.mouse_encoding = TERM_MOUSE_ENC_X10; + MOUSE(2, 1, .button = TERM_MOUSE_WHEEL_UP); expect("\033[M\140\043\042"); + + m.mouse = TERM_MOUSE_X10; + MOUSE(2, 1, .kind = TERM_MOUSE_PRESS); expect("\033[M\040\043\042"); + MOUSE(2, 1, .kind = TERM_MOUSE_RELEASE); expect(""); + + { + struct term_keyboard_state s2 = {0}; + struct term_modes m2 = { .keyboard = 1 | 2 | 8, .grid = { .x = 80, .y = 24 } }; + char b2[64]; + + struct term_key_event sh = { + .key = TERM_KEY_LSHIFT, .action = TERM_KEY_PRESS, + .mods = TERM_MOD_SHIFT, + }; + term_keyboard_update(&s2, &sh); + struct term_key_event rsh = { + .key = TERM_KEY_RSHIFT, .action = TERM_KEY_PRESS, + .mods = TERM_MOD_SHIFT, + }; + term_keyboard_update(&s2, &rsh); + + struct term_key_event rel = { + .key = TERM_KEY_LSHIFT, .action = TERM_KEY_RELEASE, + .mods = TERM_MOD_SHIFT, + }; + term_keyboard_update(&s2, &rel); + struct term_input_result r = + term_input_encode_key(&s2, &rel, &m2, b2, sizeof(b2)); + assert(r.status == TERM_INPUT_BYTES); + assert(!memcmp(b2, "\033[57441;2:3u", r.len)); + + struct term_key_event rrel = { + .key = TERM_KEY_RSHIFT, .action = TERM_KEY_RELEASE, + .mods = TERM_MOD_SHIFT, + }; + term_keyboard_update(&s2, &rrel); + r = term_input_encode_key(&s2, &rrel, &m2, b2, sizeof(b2)); + assert(!memcmp(b2, "\033[57447;1:3u", r.len)); + } + + { + char tiny[4]; + struct term_input_result r = term_input_encode_key( + &st, + &(struct term_key_event){ + .key = TERM_KEY_DELETE, .action = TERM_KEY_PRESS, + .mods = TERM_MOD_CTRL | TERM_MOD_ALT | TERM_MOD_SHIFT, + }, + &(struct term_modes){ .grid = { .x = 80, .y = 24 } }, + tiny, sizeof(tiny)); + assert(r.status == TERM_INPUT_BUFFER_FULL && r.len == 0); + } + + { + struct term_keyboard_state s5 = {0}; + struct term_key_event bad = { + .key = UINT16_MAX, .action = TERM_KEY_PRESS, + }; + term_keyboard_update(&s5, &bad); + assert(!term_keyboard_key_down(&s5, TERM_KEY_LSHIFT)); + struct term_input_result r = + term_input_encode_key(&s5, &bad, &m, out, sizeof(out)); + assert(r.status == TERM_INPUT_UNHANDLED && r.len == 0); + + struct term_key_event bad_kp = { + .key = TERM_KEY_KP_1, .keypad = UINT16_MAX, + .action = TERM_KEY_PRESS, + }; + m.app_keypad = true; + r = term_input_encode_key(&s5, &bad_kp, &m, out, sizeof(out)); + assert(r.len == 0); + m.app_keypad = false; + } + + { + struct term_key_event deferred = { + .action = TERM_KEY_PRESS, .codepoint = 'a', .text = 'a', + .text_deferred = true, + }; + struct term_input_result r = term_input_encode_key( + &st, &deferred, &(struct term_modes){0}, out, sizeof(out)); + assert(r.status == TERM_INPUT_UNHANDLED && r.len == 0); + struct term_modes kitty = { .keyboard = TERM_KEYBOARD_ALL_KEYS }; + r = term_input_encode_key(&st, &deferred, &kitty, out, sizeof(out)); + assert(r.status == TERM_INPUT_BYTES && r.len == strlen("\033[97u")); + assert(!memcmp(out, "\033[97u", r.len)); + deferred.action = TERM_KEY_RELEASE; + r = term_input_encode_key(&st, &deferred, &kitty, out, sizeof(out)); + assert(r.status == TERM_INPUT_CONSUMED && r.len == 0); + } + m.mouse = TERM_MOUSE_ANY; + m.mouse_encoding = TERM_MOUSE_ENC_SGR; + MOUSE(2, 1, .button = TERM_MOUSE_BUTTON_EXTRA_1); expect("\033[<8;3;2M"); + MOUSE(2, 1, .button = TERM_MOUSE_WHEEL_LEFT); expect("\033[<66;3;2M"); + MOUSE(2, 1, .button = TERM_MOUSE_WHEEL_RIGHT); expect("\033[<67;3;2M"); + + { + /* A host sizes the paste buffer from the text alone. */ + static const char text[] = "ls\r\n\033[31m\u202erm\x7f\t\xe4\xb8\xad"; + static const char want[] = "\033[200~ls\r[31mrm\t\xe4\xb8\xad\033[201~"; + char buf[sizeof(text) - 1 + TERM_PASTE_EXTRA]; + struct term_modes bracketed = { .bracketed_paste = true }; + struct term_input_result r = + term_input_encode_paste(&bracketed, text, sizeof(text) - 1, buf, sizeof(buf)); + assert(r.status == TERM_INPUT_BYTES && r.len == sizeof(want) - 1); + assert(!memcmp(buf, want, r.len)); + + r = term_input_encode_paste(&bracketed, text, sizeof(text) - 1, buf, r.len - 1); + assert(r.status == TERM_INPUT_BUFFER_FULL && r.len == 0); + + r = term_input_encode_paste(&bracketed, "\033\a\u2066", 5, buf, sizeof(buf)); + assert(r.status == TERM_INPUT_CONSUMED && r.len == 0); + + struct term_modes plain = {0}; + r = term_input_encode_paste(&plain, "a\nb", 3, buf, 3); + assert(r.status == TERM_INPUT_BYTES && r.len == 3 && !memcmp(buf, "a\rb", 3)); + } + puts("PASS: terminal key, mouse, and paste encoding"); + return 0; +} diff --git a/tests/meson.build b/tests/meson.build new file mode 100644 index 0000000..42c442e --- /dev/null +++ b/tests/meson.build @@ -0,0 +1,128 @@ +thread_dep = dependency('threads') +dl_dep = cc.find_library('dl', required: false) +python = find_program('python3', native: true, version: '>=3.10') +bash = find_program('bash', native: true) + +# The tests check their steps with assert, so NDEBUG must stay off. +test_c_args = ['-UNDEBUG'] + +# The host terminal may preload a library into every child. Keep it out. +test_env = environment() +test_env.set('LD_PRELOAD', '') +test_env.set('DYLD_INSERT_LIBRARIES', '') + +# [name, sources, dependencies, suites] +term_tests = [ + ['unicode-test', files('unicode/unicode_test.c'), [], []], + ['term-input-test', files('input_test.c'), [], []], + ['term-test', files('term_test.c'), [thread_dep], ['isa']], + ['write-fuzz-test', files('write_fuzz_test.c'), [], ['isa']], + ['spec-test', files('spec_test.c'), [], []], + ['model-test', files('model_test.c', 'model.c'), [], ['isa']], + ['model-row-test', files('model_row_test.c', 'model.c'), [], []], + ['gstore-test', files('gstore_test.c'), [], []], + ['hist-test', files('hist_test.c'), [], []], + ['history-text-test', files('history_text_test.c'), [thread_dep], []], + ['view-test', files('view_test.c'), [], []], + ['resize-property-test', files('resize_property_test.c'), [], []], + ['term-resize-failure-test', files('resize_failure_test.c'), [dl_dep], ['extlib']], + ['dcs-test', files('dcs_test.c'), [], []], + ['osc-test', files('osc_test.c'), [], []], +] +test_helpers_inc = include_directories('helpers') + +term_test_exes = {} +foreach entry : term_tests + exe = executable(entry[0], entry[1], + c_args: test_c_args, + include_directories: test_helpers_inc, + dependencies: [term_internal_dep] + entry[2], + ) + term_test_exes += {entry[0]: exe} + test(entry[0], exe, + args: entry[0] == 'write-fuzz-test' ? [get_option('write_fuzz_seeds').to_string()] : [], + env: test_env, + suite: ['unit'] + entry[3], + timeout: 180, + ) +endforeach + +# Without -march, the core runs the best kernels the CPU supports. Run the +# tests again on the lower levels, so each one is covered. +if host_machine.cpu_family() == 'x86_64' and cc.get_argument_syntax() == 'gcc' and \ + cc.get_define('__AVX2__', args: get_option('arch_flag') != '' ? [get_option('arch_flag')] : []) == '' + foreach level : ['base', 'avx2'] + level_env = test_env + level_env.set('TERM_HEADLESS_ISA', level) + foreach name : ['term-test', 'spec-test', 'model-test', 'view-test', 'term-input-test', + 'hist-test', 'history-text-test'] + test(name + '-' + level, term_test_exes[name], env: level_env, + suite: ['unit', 'isa'], timeout: 180) + endforeach + endforeach +endif + +# The amalgamation is the core as one term.c, the way a host builds it. The +# tests that use only term.h run again against it. +term_amalgamation = custom_target( + 'term-amalgamation', + output: ['term.c', 'term.h'], + depfile: 'term-amalgamation.d', + command: [python, files('../tools/amalgamate.py'), + meson.project_source_root(), '@OUTDIR@', '@DEPFILE@'], +) +term_amalgamation_lib = static_library( + 'term-amalgamation', + term_amalgamation[0], + dependencies: [m_dep], + override_options: ['c_std=c11', 'warning_level=3', 'werror=true'], + build_by_default: false, +) +foreach entry : term_tests + if entry[0] in ['term-test', 'spec-test', 'view-test', 'dcs-test', 'osc-test', + 'history-text-test', 'term-input-test', 'model-test', 'model-row-test', + 'resize-property-test'] + test(entry[0] + '-amalgamation', + executable(entry[0] + '-amalgamation', entry[1], + c_args: test_c_args, + include_directories: [term_inc, test_helpers_inc], + link_with: term_amalgamation_lib, + dependencies: [m_dep] + entry[2], + build_by_default: false), + env: test_env, suite: ['unit', 'amalgamation'], timeout: 180) + endif +endforeach + +test('term-symbols-test', bash, + args: [files('symbols_test.sh'), term_headless_lib.get_static_lib(), + meson.project_source_root() / 'include' / 'term', + term_headless_lib.get_shared_lib()], + suite: 'unit', +) +test('term-header-test', bash, + args: [files('header_test.sh'), meson.project_source_root()], + suite: 'unit', +) +test('term-cross-test', bash, + args: [files('cross_test.sh'), meson.project_source_root()], + suite: 'portability', + timeout: 600, +) + +if get_option('kitty_tests') + if host_machine.system() != 'linux' or get_option('b_sanitize') != 'none' + error('Kitty tests require an unsanitized Linux build') + endif + kitty_bridge = shared_library('term', 'extern/kitty/screen_bridge.c', + dependencies: term_internal_dep, + ) + kitty_env = environment() + kitty_env.set('WSAP_KITTY_LIB', kitty_bridge.full_path()) + test('kitty-test', python, + args: files('extern/kitty/parser.py'), + depends: kitty_bridge, + env: kitty_env, + suite: 'kitty', + is_parallel: false, + ) +endif diff --git a/tests/model.c b/tests/model.c new file mode 100644 index 0000000..4fd3737 --- /dev/null +++ b/tests/model.c @@ -0,0 +1,2105 @@ +/* + * Reference model of the terminal. See model.h. + * + * Sources of truth, in order: + * - xterm and ECMA-48 for controls, escape sequences, and modes. + * - Unicode for UTF-8: each maximal invalid subpart becomes one U+FFFD. + * - This terminal's design where xterm has no rule: reflow on resize, the + * row length that reflow keeps, history, and grapheme clusters. + * + * The model is slow on purpose. It has no lazy rows, no rings, and no SIMD. + * + * Known differences from xterm follow. For each one the model matches the + * current terminal, and the macro names the phase that removes it. + */ +#include "model.h" +#include "term/unicode/unicode.h" + +#include <limits.h> +#include <stdarg.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> + +/* Phase 6: the history limit counts rows. It must count logical lines. */ +#define KNOWN_HISTORY_COUNTS_ROWS 0 +/* Phase 6: a column resize on the alternate screen also cuts the saved + * normal screen and the history to the new width. */ +#define KNOWN_ALT_RESIZE_CUTS_NORMAL 0 + +#define MAX_TABS 4096 +#define STR_KEEP 1024 + +struct cell { + uint32_t cp[GRAPHEME_MAX_LEN]; + uint8_t n, width; + uint16_t attrs; + term_color fg, bg; + bool pad; /* the blank that a wide glyph left at a row end */ +}; + +/* + * A row. len is the length that reflow keeps: it covers every cell that + * was written or erased in a color, and a default erase at the end of the + * row shortens it. Cells at or after len are default blanks. + */ +struct row { + int len; + bool wrapped; /* autowrap continued this row on the next one */ + struct cell c[]; +}; + +struct pen { + term_color fg, bg, ul; + uint16_t attrs; /* TERM_ATTR_* style bits and the underline style */ + bool inverse; +}; + +/* What DECSC saves. */ +struct saved { + int x, y; + bool pending, origin; + struct pen pen; + char g[2]; + int gl; +}; + +struct keyboard { + unsigned flags, saved[32], depth; +}; + +enum { GROUND, ESC, CSI, STR, STR_ESC, CHARSET, HASH, DOCS }; +enum { OSC, DCS, IGNORE }; + +struct model { + int cols, rows, limit; + struct row **scr; + struct row **hist; + int hist_len, hist_cap, hist_lines; + + /* The normal screen while the alternate screen is active. */ + bool alternate; + struct row **normal; + int normal_cols, normal_rows; + struct saved normal_cursor, normal_saved; + bool normal_visible, normal_blink; + int normal_style; + + int x, y; + bool pending; + struct pen pen; + char g[2]; + int gl; + struct saved saved; + bool autowrap, origin, insert, lnm, reverse, app_cursor, app_keypad; + bool backarrow, bracketed, focus, sync, visible, blink; + int style, mouse, mouse_enc, decsace; + int top, bottom; + bool tabs[MAX_TABS]; + struct keyboard keyboard[2]; + /* The last printed glyph, for REP and cluster joins. */ + uint32_t last[GRAPHEME_MAX_LEN]; + int last_n; + int prompt; /* screen row of the last OSC 133;A, or -1 */ + unsigned flags; + + int state, str_kind; + char slot; + int param[16], nparam, csi_phase; + uint16_t sub; + char prefix, inter; + char str[STR_KEEP]; + size_t str_len; + + int need; + uint32_t acc; + uint8_t lo, hi; + + char *reply; + size_t reply_len, reply_cap; + struct model_cell **out; + int out_cap, out_cols; +}; + +/* ---- cells and rows ---- */ + +static struct cell blank(term_color fg, term_color bg) +{ + return (struct cell){ .cp = { ' ' }, .n = 1, .width = 1, .fg = fg, .bg = bg }; +} + +static struct cell default_blank(void) +{ + return blank(TERM_COLOR_FG, TERM_COLOR_BG); +} + +static struct row *row_new(int cols) +{ + struct row *r = malloc(sizeof(*r) + (size_t)cols * sizeof(struct cell)); + if (!r) abort(); + r->len = 0; + r->wrapped = false; + for (int x = 0; x < cols; x++) r->c[x] = default_blank(); + return r; +} + +/* Cut len to n. The cells after it become default blanks. */ +static void row_cut(struct row *r, int n, int cols) +{ + if (n >= r->len) return; + for (int x = n; x < r->len && x < cols; x++) r->c[x] = default_blank(); + r->len = n; +} + +static bool pristine(const struct model *m) +{ + return m->pen.fg == TERM_COLOR_FG && m->pen.bg == TERM_COLOR_BG; +} + +static bool is_ri(uint32_t cp) +{ + return cp - 0x1F1E6u <= 0x19u && grapheme_regional_indicator(cp); +} + +/* ---- replies ---- */ + +static void __attribute__((format(printf, 2, 3))) reply(struct model *m, const char *fmt, ...) +{ + char tmp[256]; + va_list args; + va_start(args, fmt); + int n = vsnprintf(tmp, sizeof(tmp), fmt, args); + va_end(args); + if (n < 0) return; + if (m->reply_len + (size_t)n + 1 > m->reply_cap) { + m->reply_cap = (m->reply_len + (size_t)n + 1) * 2; + m->reply = realloc(m->reply, m->reply_cap); + if (!m->reply) abort(); + } + memcpy(m->reply + m->reply_len, tmp, (size_t)n); + m->reply_len += (size_t)n; +} + +/* ---- history ---- */ + +static int history_limit(int requested) +{ + return requested == 0 ? TERM_SCROLLBACK_DEFAULT : requested < 0 ? 0 : requested; +} + +static void hist_reflow_tail(struct model *m); + +static void hist_push(struct model *m, struct row *r) +{ + if (!m->limit) { + free(r); + return; + } + if (KNOWN_HISTORY_COUNTS_ROWS) { + if (m->hist_len == m->limit) { + free(m->hist[0]); + memmove(m->hist, m->hist + 1, (size_t)(m->hist_len - 1) * sizeof(*m->hist)); + m->hist_len--; + } + } else { + bool new_line = (m->hist_len == 0 || !m->hist[m->hist_len - 1]->wrapped); + if (new_line) { + if (m->hist_lines == m->limit) { + int evict_rows = 1; + while (evict_rows < m->hist_len && m->hist[evict_rows - 1]->wrapped) + evict_rows++; + for (int i = 0; i < evict_rows; i++) + free(m->hist[i]); + memmove(m->hist, m->hist + evict_rows, + (size_t)(m->hist_len - evict_rows) * sizeof(*m->hist)); + m->hist_len -= evict_rows; + m->hist_lines--; + } + m->hist_lines++; + } + } + if (m->hist_len == m->hist_cap) { + m->hist_cap = m->hist_cap ? m->hist_cap * 2 : 64; + m->hist = realloc(m->hist, (size_t)m->hist_cap * sizeof(*m->hist)); + if (!m->hist) abort(); + } + m->hist[m->hist_len++] = r; + if (!KNOWN_HISTORY_COUNTS_ROWS) + hist_reflow_tail(m); +} + +static void term_hist_clear(struct model *m) +{ + for (int i = 0; i < m->hist_len; i++) free(m->hist[i]); + m->hist_len = 0; + m->hist_lines = 0; +} + +/* ---- scrolling ---- */ + +static void erase_row(struct model *m, int y); + +/* Scroll the whole screen up one row. On the normal screen the top row + * goes to history. */ +static void scroll_up_full(struct model *m) +{ + struct row *top = m->scr[0]; + memmove(m->scr, m->scr + 1, (size_t)(m->rows - 1) * sizeof(*m->scr)); + m->scr[m->rows - 1] = row_new(m->cols); + erase_row(m, m->rows - 1); + if (m->alternate) { + free(top); + return; + } + if (m->prompt > 0) m->prompt--; + hist_push(m, top); +} + +static void scroll_region(struct model *m, int top, int bottom, int n, bool down, bool history) +{ + int height = bottom - top + 1; + if (n > height) n = height; + if (!down && history && top == 0 && bottom == m->rows - 1) { + for (int i = 0; i < n; i++) scroll_up_full(m); + return; + } + /* A region at the top of the normal screen saves the rows that leave + * it. The rows below the region stay in place. */ + if (!down && history && top == 0 && !m->alternate) { + for (int i = 0; i < n; i++) { + scroll_up_full(m); + scroll_region(m, bottom, m->rows - 1, 1, true, false); + } + return; + } + for (int i = 0; i < n; i++) { + if (down) { + free(m->scr[bottom]); + memmove(m->scr + top + 1, m->scr + top, (size_t)(height - 1) * sizeof(*m->scr)); + m->scr[top] = row_new(m->cols); + erase_row(m, top); + } else { + free(m->scr[top]); + memmove(m->scr + top, m->scr + top + 1, (size_t)(height - 1) * sizeof(*m->scr)); + m->scr[bottom] = row_new(m->cols); + erase_row(m, bottom); + } + } + if (!m->alternate && m->prompt >= top && m->prompt <= bottom) { + m->prompt += down ? n : -n; + if (m->prompt < top || m->prompt > bottom) m->prompt = -1; + } +} + +static void line_feed(struct model *m) +{ + if (m->y == m->bottom) { + scroll_region(m, m->top, m->bottom, 1, false, true); + } else if (m->y < m->rows - 1) { + m->y++; + } +} + +static void reverse_index(struct model *m) +{ + if (m->y == m->top) + scroll_region(m, m->top, m->bottom, 1, true, false); + else if (m->y > 0) + m->y--; +} + +/* ---- erasing ---- */ + +/* An erase that covers half of a wide glyph erases all of it. Erased cells + * take the pen colors without inverse and lose their attributes. */ +static void erase_cells(struct model *m, int y, int from, int to) +{ + struct row *r = m->scr[y]; + if (to > m->cols) to = m->cols; + bool plain = pristine(m); + if (plain && to > r->len) to = r->len; + if (from >= to) return; + if (from > 0 && r->c[from].width == 0) from--; + if (to < r->len && r->c[to].width == 0) to++; + if (to >= m->cols) r->wrapped = false; + if (!plain) { + for (int x = from; x < to; x++) r->c[x] = blank(m->pen.fg, m->pen.bg); + if (to > r->len) r->len = to; + return; + } + if (to < r->len) { + for (int x = from; x < to; x++) r->c[x] = default_blank(); + return; + } + row_cut(r, from, m->cols); + if (from == 0) r->wrapped = false; +} + +static void erase_row(struct model *m, int y) +{ + erase_cells(m, y, 0, m->cols); +} + +/* The last screen row that shows a glyph other than a space. */ +static int last_text_row(const struct model *m) +{ + for (int y = m->rows - 1; y >= 0; y--) + for (int x = 0; x < m->cols; x++) { + const struct cell *c = &m->scr[y]->c[x]; + if (c->n > 1 || (c->n == 1 && c->cp[0] > ' ')) return y; + } + return -1; +} + +static void erase_display(struct model *m, int mode) +{ + if (mode == 3) { + term_hist_clear(m); + return; + } + if (mode == 2) { + /* The normal screen keeps its text: rows up to the last one with + * text move to history first. */ + int last = m->alternate ? -1 : last_text_row(m); + for (int i = 0; i <= last; i++) scroll_up_full(m); + for (int y = 0; y < m->rows; y++) erase_row(m, y); + } else if (mode == 0) { + erase_cells(m, m->y, m->x, m->cols); + for (int y = m->y + 1; y < m->rows; y++) erase_row(m, y); + } else if (mode == 1) { + for (int y = 0; y < m->y; y++) erase_row(m, y); + erase_cells(m, m->y, 0, m->x + 1); + } +} + +static void erase_line(struct model *m, int mode) +{ + if (mode == 0) erase_cells(m, m->y, m->x, m->cols); + else if (mode == 1) erase_cells(m, m->y, 0, m->x + 1); + else if (mode == 2) erase_cells(m, m->y, 0, m->cols); +} + +/* ---- wide glyph repair ---- */ + +/* A write to [from, to) splits the wide glyphs at its edges. Their other + * halves become blanks that keep their colors. */ +static void split_edges(struct model *m, struct row *r, int from, int to) +{ + if (from >= to) return; + if (from > 0 && from < r->len && r->c[from].width == 0) + r->c[from - 1] = blank(r->c[from - 1].fg, r->c[from - 1].bg); + if (to < r->len && r->c[to].width == 0) + r->c[to] = blank(r->c[to].fg, r->c[to].bg); + (void)m; +} + +/* Blank every half of a wide glyph in [from, to) that lost its partner. */ +static void repair_row(struct model *m, struct row *r, int from, int to) +{ + if (from < 0) from = 0; + if (to > m->cols) to = m->cols; + for (int x = from; x < to; x++) { + struct cell *c = &r->c[x]; + if (c->width == 0 && (x == 0 || r->c[x - 1].width != 2)) + *c = blank(c->fg, c->bg); + if (c->width == 2 && m->cols > 1 && (x + 1 == m->cols || r->c[x + 1].width != 0)) + *c = blank(c->fg, c->bg); + } +} + +/* Writing at x past len first fills the gap with default blanks. */ +static void fill_gap(struct row *r, int x) +{ + if (x > r->len) r->len = x; +} + +/* ---- printing ---- */ + +static uint16_t presentation(const uint32_t *cps, int n) +{ + bool pictographic = unicode_extended_pictographic(cps[0]); + bool emoji = false; + for (int i = 1; i < n; i++) { + if (cps[i] == 0xFE0F) return TERM_ATTR_EMOJI; + if (cps[i] == 0xFE0E) return TERM_ATTR_TEXT; + if (cps[i] == 0x20E3 || grapheme_regional_indicator(cps[i]) || + (pictographic && (cps[i] == 0x200D || grapheme_emoji_modifier(cps[i]) || + (cps[i] >= 0xE0020 && cps[i] <= 0xE007F)))) + emoji = true; + } + return emoji ? TERM_ATTR_EMOJI : 0; +} + +static void wrap_to_next_row(struct model *m) +{ + m->scr[m->y]->wrapped = true; + m->x = 0; + line_feed(m); + m->pending = false; +} + +/* Put one glyph at the cursor with the pen colors and the given style. */ +static void put_glyph(struct model *m, const uint32_t *cps, int n, uint16_t attrs) +{ + if (n > 1) + attrs = (uint16_t)((attrs & ~(TERM_ATTR_EMOJI | TERM_ATTR_TEXT)) | presentation(cps, n)); + int width = unicode_width(cps[0]) == 2 ? 2 : 1; + if (attrs & TERM_ATTR_EMOJI) width = 2; + if (attrs & TERM_ATTR_TEXT) width = 1; + if (m->pending) wrap_to_next_row(m); + int right = m->cols - 1; + struct row *r = m->scr[m->y]; + /* A wide glyph that does not fit wraps and leaves a pad blank. */ + if (width == 2 && m->cols > 1 && m->x == right && m->autowrap) { + fill_gap(r, m->x); + split_edges(m, r, m->x, right + 1); + r->c[m->x] = default_blank(); + r->c[m->x].pad = true; + r->len = m->cols; + wrap_to_next_row(m); + r = m->scr[m->y]; + } + int x = m->x; + int span = width > right + 1 - x ? 1 : width; + fill_gap(r, x); + if (m->insert) { + int move = right + 1 - x - span; + if (move > 0) memmove(r->c + x + span, r->c + x, (size_t)move * sizeof(struct cell)); + r->len = r->len + span > m->cols ? m->cols : r->len + span; + repair_row(m, r, right, right + 1); + } + split_edges(m, r, x, x + span); + struct cell c = { + .n = (uint8_t)n, .width = (uint8_t)span, .attrs = attrs, + .fg = m->pen.inverse ? m->pen.bg : m->pen.fg, + .bg = m->pen.inverse ? m->pen.fg : m->pen.bg, + }; + memcpy(c.cp, cps, (size_t)n * sizeof(*cps)); + r->c[x] = c; + if (span == 2) { + r->c[x + 1] = c; + r->c[x + 1].n = 0; + r->c[x + 1].width = 0; + memset(r->c[x + 1].cp, 0, sizeof(c.cp)); + } + if (x + span > r->len) r->len = x + span; + if (x + span <= right) { + m->x = x + span; + } else { + m->x = right; + m->pending = m->autowrap; + } +} + +/* Marks, emoji modifiers, and spacing marks extend the glyph before them. */ +static bool joins(uint32_t cp, int width) +{ + if (cp < 0x300) return false; + return width == 0 || grapheme_emoji_modifier(cp) || + (width == 1 && unicode_spacing_mark(cp)); +} + +static bool cluster_joins(const uint32_t *cps, int n, uint32_t cp, int width) +{ + if (joins(cp, width)) return true; + if (cps[n - 1] == 0x200D) return unicode_extended_pictographic(cp); + return n == 1 && is_ri(cp) && is_ri(cps[0]); +} + +/* + * Try to add cp to the glyph left of the cursor. True when cp was used: + * joined, or dropped because it has no glyph to join. The glyph keeps its + * colors and style and is placed again, so it can grow to two cells. + */ +static bool extend_cluster(struct model *m, uint32_t cp, int width) +{ + bool after_join = m->last_n > 1 || (m->last_n == 1 && m->last[0] == 0x200D); + if (!(width == 0 || after_join || is_ri(cp) || joins(cp, width))) + return false; + struct row *r = m->scr[m->y]; + int x = m->pending ? m->cols - 1 : m->x - 1; + struct cell *prev = NULL; + if (x >= 0 && x < r->len) { + if (x > 0 && r->c[x].width == 0) x--; + prev = &r->c[x]; + } + if (!prev || !(prev->n > 1 || (prev->n == 1 && prev->cp[0] > ' '))) + return joins(cp, width); + if (!cluster_joins(prev->cp, prev->n, cp, width)) + return false; + if (prev->n >= GRAPHEME_MAX_LEN) + return true; + struct cell old = *prev; + uint32_t cps[GRAPHEME_MAX_LEN]; + memcpy(cps, old.cp, (size_t)old.n * sizeof(*cps)); + cps[old.n] = cp; + int n = old.n + 1; + int span = old.width == 2 && x + 1 < m->cols ? 2 : 1; + for (int i = 0; i < span; i++) r->c[x + i] = blank(old.fg, old.bg); + if (r->len == x + span) row_cut(r, x, m->cols); + m->x = x; + m->pending = false; + memcpy(m->last, cps, (size_t)n * sizeof(*cps)); + m->last_n = n; + put_glyph(m, cps, n, (uint16_t)(old.attrs & ~(TERM_ATTR_EMOJI | TERM_ATTR_TEXT))); + r = m->scr[m->y]; + x = m->pending ? m->cols - 1 : m->x - 1; + if (x > 0 && r->c[x].width == 0) x--; + r->c[x].fg = old.fg; + r->c[x].bg = old.bg; + if (r->c[x].width == 2 && x + 1 < m->cols) { + r->c[x + 1].fg = old.fg; + r->c[x + 1].bg = old.bg; + } + return true; +} + +static void esc_dispatch(struct model *m, unsigned char ch); + +static void emit(struct model *m, uint32_t cp) +{ + static const uint32_t dec_graphics[] = { + 0x00a0, + 0x25c6, 0x2592, 0x2409, 0x240c, 0x240d, 0x240a, 0x00b0, 0x00b1, + 0x2424, 0x240b, 0x2518, 0x2510, 0x250c, 0x2514, 0x253c, 0x23ba, + 0x23bb, 0x2500, 0x23bc, 0x23bd, 0x251c, 0x2524, 0x2534, 0x252c, + 0x2502, 0x2264, 0x2265, 0x03c0, 0x2260, 0x00a3, 0x00b7, + }; + /* C1 controls in UTF-8 form act as their ESC forms. */ + if (cp >= 0x80 && cp <= 0x9f) { + static const char c1[32] = { + [4] = 'D', [5] = 'E', [8] = 'H', [13] = 'M', + [16] = 'P', [27] = '[', [28] = '\\', [29] = ']', + }; + if (c1[cp - 0x80]) { + m->state = GROUND; + esc_dispatch(m, (unsigned char)c1[cp - 0x80]); + } + return; + } + if (m->g[m->gl] == '0' && cp >= '_' && cp <= '~') + cp = dec_graphics[cp - '_']; + m->last[0] = cp; + m->last_n = 1; + put_glyph(m, &cp, 1, m->pen.attrs); +} + +static void print_cp(struct model *m, uint32_t cp) +{ + if (cp < 0x80) { + emit(m, cp); + return; + } + int width = unicode_width(cp); + if (!extend_cluster(m, cp, width)) + emit(m, cp); +} + +/* ---- cursor ---- */ + +static int top_limit(const struct model *m) { return m->origin ? m->top : 0; } +static int bottom_limit(const struct model *m) { return m->origin ? m->bottom : m->rows - 1; } + +static void move_to(struct model *m, int x, int y) +{ + m->pending = false; + m->x = x < 0 ? 0 : x >= m->cols ? m->cols - 1 : x; + m->y = y < 0 ? 0 : y >= m->rows ? m->rows - 1 : y; +} + +/* CUU, CUD, CNL, CPL, and VPR stop at a margin when they start inside the + * scroll region (xterm, DEC STD 070). */ +static void move_v(struct model *m, int n, bool col0) +{ + int lo = 0, hi = m->rows - 1; + if (m->origin || (m->y >= m->top && m->y <= m->bottom)) { + lo = m->top; + hi = m->bottom; + } else if (m->y < m->top) { + hi = m->bottom; + } else { + lo = m->top; + } + long y = (long)m->y + n; + if (n < 0 && m->y >= lo && y < lo) y = lo; + if (n > 0 && m->y <= hi && y > hi) y = hi; + move_to(m, col0 ? 0 : m->x, (int)(y < 0 ? 0 : y > m->rows - 1 ? m->rows - 1 : y)); +} + +static void move_h(struct model *m, int n) +{ + long x = (long)m->x + n; + move_to(m, (int)(x < 0 ? 0 : x > m->cols - 1 ? m->cols - 1 : x), m->y); +} + +static struct saved save_cursor(const struct model *m) +{ + return (struct saved){ + .x = m->x, .y = m->y, .pending = m->pending, .origin = m->origin, + .pen = m->pen, .g = { m->g[0], m->g[1] }, .gl = m->gl, + }; +} + +static void restore_cursor(struct model *m, const struct saved *s) +{ + m->pen = s->pen; + m->origin = s->origin; + m->g[0] = s->g[0]; + m->g[1] = s->g[1]; + m->gl = s->gl; + move_to(m, s->x, s->y); + m->pending = s->pending; +} + +static struct saved initial_saved(void) +{ + return (struct saved){ + .pen = { .fg = TERM_COLOR_FG, .bg = TERM_COLOR_BG }, + .g = { 'B', 'B' }, + }; +} + +static void tabs_default(struct model *m) +{ + memset(m->tabs, 0, sizeof(m->tabs)); + for (int c = 8; c < MAX_TABS; c += 8) m->tabs[c] = true; +} + +/* ---- insert and delete ---- */ + +static void delete_chars(struct model *m, int n) +{ + struct row *r = m->scr[m->y]; + int x = m->x, cols = m->cols; + if (n > cols - x) n = cols - x; + int len = r->len; + split_edges(m, r, x, x + n); + if (len > x + n) { + memmove(r->c + x, r->c + x + n, (size_t)(len - x - n) * sizeof(struct cell)); + for (int i = len - n; i < len; i++) r->c[i] = default_blank(); + r->len = len - n; + } else if (len > x) { + row_cut(r, x, cols); + } + if (!pristine(m)) { + int from = cols - n < x ? x : cols - n; + fill_gap(r, from); + for (int i = from; i < cols; i++) r->c[i] = blank(m->pen.fg, m->pen.bg); + r->len = cols; + } + r->wrapped = false; + repair_row(m, r, x - 1, len); +} + +static void insert_chars(struct model *m, int n) +{ + struct row *r = m->scr[m->y]; + int x = m->x, cols = m->cols; + if (n > cols - x) n = cols - x; + fill_gap(r, x); + int len = r->len; + if (x > 0 && r->c[x].width == 0) { + r->c[x - 1] = blank(m->pen.fg, m->pen.bg); + r->c[x] = blank(m->pen.fg, m->pen.bg); + } + if (len > x) { + int move = len - x; + if (move > cols - x - n) move = cols - x - n; + if (move > 0) memmove(r->c + x + n, r->c + x, (size_t)move * sizeof(struct cell)); + } + for (int i = x; i < x + n; i++) r->c[i] = blank(m->pen.fg, m->pen.bg); + int new_len = len + n > cols ? cols : len + n; + if (pristine(m) && len <= x) new_len = len; + else if (new_len < x + n) new_len = x + n; + for (int i = new_len; i < cols; i++) r->c[i] = default_blank(); + r->len = new_len; + r->wrapped = false; + int end = x + n + 1 > len + 1 ? x + n + 1 : len + 1; + repair_row(m, r, x - 1, end); +} + +/* ---- SGR ---- */ + +static void pen_reset(struct pen *p) +{ + *p = (struct pen){ .fg = TERM_COLOR_FG, .bg = TERM_COLOR_BG }; +} + +static void set_underline(uint16_t *attrs, int style) +{ + if (style > 5) style = 1; + *attrs = (uint16_t)((*attrs & ~TERM_ATTR_UNDERLINE_MASK) | + (unsigned)style << TERM_ATTR_UNDERLINE_SHIFT); +} + +/* 38, 48, and 58. Returns the parameters it used after the first. */ +static int extended_color(const struct model *m, int i, term_color *color, bool *ok) +{ + *ok = false; + if (i + 1 >= m->nparam) return 0; + if (i + 2 < m->nparam && m->param[i + 1] == 5) { + int index = m->param[i + 2]; + if (index >= 0 && index <= 255) { + *color = term_color_index(index); + *ok = true; + } + return 2; + } + if (m->param[i + 1] != 2) return 0; + int colon = 0; + while (i + 2 + colon < m->nparam && i + 2 + colon < 16 && + (m->sub & (1u << (i + 2 + colon)))) + colon++; + int skip = colon >= 4 ? 1 : 0; + if (i + 4 + skip >= m->nparam) return 0; + *color = term_color_rgb((uint8_t)m->param[i + 2 + skip], (uint8_t)m->param[i + 3 + skip], + (uint8_t)m->param[i + 4 + skip]); + *ok = true; + return 4 + skip; +} + +static void sgr(struct model *m) +{ + struct pen *p = &m->pen; + if (m->nparam == 0) { + pen_reset(p); + return; + } + for (int i = 0; i < m->nparam; i++) { + int v = m->param[i]; + term_color color; + bool ok; + switch (v) { + case 0: pen_reset(p); break; + case 1: p->attrs |= TERM_ATTR_BOLD; break; + case 2: p->attrs |= TERM_ATTR_FAINT; break; + case 3: p->attrs |= TERM_ATTR_ITALIC; break; + case 4: { + int style = 1; + if (i + 1 < m->nparam && (m->sub & (1u << (i + 1)))) + style = m->param[++i]; + set_underline(&p->attrs, style); + break; + } + case 7: p->inverse = true; break; + case 9: p->attrs |= TERM_ATTR_STRIKE; break; + case 21: set_underline(&p->attrs, 2); break; + case 22: p->attrs &= (uint16_t)~(TERM_ATTR_BOLD | TERM_ATTR_FAINT); break; + case 23: p->attrs &= (uint16_t)~TERM_ATTR_ITALIC; break; + case 24: set_underline(&p->attrs, 0); break; + case 27: p->inverse = false; break; + case 29: p->attrs &= (uint16_t)~TERM_ATTR_STRIKE; break; + case 38: i += extended_color(m, i, &color, &ok); if (ok) p->fg = color; break; + case 39: p->fg = TERM_COLOR_FG; break; + case 48: i += extended_color(m, i, &color, &ok); if (ok) p->bg = color; break; + case 49: p->bg = TERM_COLOR_BG; break; + case 53: p->attrs |= TERM_ATTR_OVERLINE; break; + case 55: p->attrs &= (uint16_t)~TERM_ATTR_OVERLINE; break; + case 58: i += extended_color(m, i, &color, &ok); if (ok) p->ul = color; break; + case 59: p->ul = 0; break; + default: + if (v >= 30 && v <= 37) p->fg = term_color_index(v - 30); + else if (v >= 40 && v <= 47) p->bg = term_color_index(v - 40); + else if (v >= 90 && v <= 97) p->fg = term_color_index(v - 90 + 8); + else if (v >= 100 && v <= 107) p->bg = term_color_index(v - 100 + 8); + break; + } + } +} + +/* The DECRQSS reply body for SGR. */ +static void sgr_report(const struct model *m, char *out, size_t cap) +{ + const struct pen *p = &m->pen; + size_t n = 0; + out[0] = 0; +#define ADD(...) do { int k = snprintf(out + n, cap - n, __VA_ARGS__); if (k > 0) n += (size_t)k; } while (0) + int ul = (int)((p->attrs & TERM_ATTR_UNDERLINE_MASK) >> TERM_ATTR_UNDERLINE_SHIFT); + if (!(p->attrs & (TERM_ATTR_BOLD | TERM_ATTR_FAINT | TERM_ATTR_ITALIC | TERM_ATTR_STRIKE | + TERM_ATTR_OVERLINE | TERM_ATTR_UNDERLINE_MASK)) && + !p->inverse && p->fg == TERM_COLOR_FG && p->bg == TERM_COLOR_BG && !p->ul) + return; + ADD("0"); + if (p->attrs & TERM_ATTR_BOLD) ADD(";1"); + if (p->attrs & TERM_ATTR_FAINT) ADD(";2"); + if (p->attrs & TERM_ATTR_ITALIC) ADD(";3"); + if (ul == 1) ADD(";4"); + else if (ul) ADD(";4:%d", ul); + if (p->inverse) ADD(";7"); + if (p->attrs & TERM_ATTR_STRIKE) ADD(";9"); + if (p->attrs & TERM_ATTR_OVERLINE) ADD(";53"); + const term_color colors[3] = { p->fg, p->bg, p->ul }; + const bool set[3] = { p->fg != TERM_COLOR_FG, p->bg != TERM_COLOR_BG, p->ul != 0 }; + for (int k = 0; k < 3; k++) { + if (!set[k]) continue; + int base = 30 + 10 * k; + term_color c = colors[k]; + if (term_color_is_slot(c) && (c & 0x1FFu) < 256) { + int i = (int)(c & 0xFFu); + if (i < 16 && k < 2) ADD(";%d", i < 8 ? base + i : base + 60 + i - 8); + else ADD(";%d:5:%d", base + 8, i); + } else { + ADD(";%d:2:%d:%d:%d", base + 8, (int)((c >> 16) & 0xff), + (int)((c >> 8) & 0xff), (int)(c & 0xff)); + } + } +#undef ADD +} + +/* DECCARA: change attributes in a rectangle, or in a stream of cells when + * DECSACE is not 2. A rectangle edge that splits a wide glyph changes both + * of its halves. */ +static void cara_cell(struct model *m, struct cell *c) +{ + if (m->nparam <= 4) { + c->fg = TERM_COLOR_FG; + c->bg = TERM_COLOR_BG; + c->attrs &= TERM_ATTR_EMOJI | TERM_ATTR_TEXT | TERM_ATTR_LINK; + return; + } + for (int i = 4; i < m->nparam; i++) { + int v = m->param[i]; + term_color color; + bool ok; + switch (v) { + case 0: + c->fg = TERM_COLOR_FG; + c->bg = TERM_COLOR_BG; + c->attrs &= TERM_ATTR_EMOJI | TERM_ATTR_TEXT | TERM_ATTR_LINK; + break; + case 1: c->attrs |= TERM_ATTR_BOLD; break; + case 2: c->attrs |= TERM_ATTR_FAINT; break; + case 3: c->attrs |= TERM_ATTR_ITALIC; break; + case 4: { + int style = 1; + if (i + 1 < m->nparam && (m->sub & (1u << (i + 1)))) + style = m->param[++i]; + set_underline(&c->attrs, style); + break; + } + case 7: { term_color t = c->fg; c->fg = c->bg; c->bg = t; break; } + case 9: c->attrs |= TERM_ATTR_STRIKE; break; + case 21: set_underline(&c->attrs, 2); break; + case 22: c->attrs &= (uint16_t)~(TERM_ATTR_BOLD | TERM_ATTR_FAINT); break; + case 23: c->attrs &= (uint16_t)~TERM_ATTR_ITALIC; break; + case 24: set_underline(&c->attrs, 0); break; + case 29: c->attrs &= (uint16_t)~TERM_ATTR_STRIKE; break; + case 38: i += extended_color(m, i, &color, &ok); if (ok) c->fg = color; break; + case 39: c->fg = TERM_COLOR_FG; break; + case 48: i += extended_color(m, i, &color, &ok); if (ok) c->bg = color; break; + case 49: c->bg = TERM_COLOR_BG; break; + case 53: c->attrs |= TERM_ATTR_OVERLINE; break; + case 55: c->attrs &= (uint16_t)~TERM_ATTR_OVERLINE; break; + case 58: i += extended_color(m, i, &color, &ok); break; + default: + if (v >= 30 && v <= 37) c->fg = term_color_index(v - 30); + else if (v >= 40 && v <= 47) c->bg = term_color_index(v - 40); + else if (v >= 90 && v <= 97) c->fg = term_color_index(v - 90 + 8); + else if (v >= 100 && v <= 107) c->bg = term_color_index(v - 100 + 8); + break; + } + } +} + +static void cara_row(struct model *m, int y, int x0, int x1) +{ + struct row *r = m->scr[y]; + if (x0 > 0 && r->c[x0].width == 0) x0--; + if (x1 + 1 < m->cols && r->c[x1].width == 2) x1++; + fill_gap(r, x1 + 1); + for (int x = x0; x <= x1; x++) cara_cell(m, &r->c[x]); +} + +static void deccara(struct model *m) +{ + int top = m->param[0] > 0 ? m->param[0] - 1 : 0; + int left = m->nparam > 1 && m->param[1] > 0 ? m->param[1] - 1 : 0; + int bottom = m->nparam > 2 && m->param[2] > 0 ? m->param[2] - 1 : m->rows - 1; + int right = m->nparam > 3 && m->param[3] > 0 ? m->param[3] - 1 : m->cols - 1; + if (top > bottom || left > right || top >= m->rows || left >= m->cols) return; + if (bottom >= m->rows) bottom = m->rows - 1; + if (right >= m->cols) right = m->cols - 1; + if (m->decsace == 2) { + for (int y = top; y <= bottom; y++) cara_row(m, y, left, right); + return; + } + cara_row(m, top, left, top == bottom ? right : m->cols - 1); + for (int y = top + 1; y < bottom; y++) cara_row(m, y, 0, m->cols - 1); + if (bottom > top) cara_row(m, bottom, 0, right); +} + +/* ---- screens and resize ---- */ + +static void free_rows(struct row **rows, int n) +{ + if (!rows) return; + for (int i = 0; i < n; i++) free(rows[i]); + free(rows); +} + +static struct row **rows_new(int n, int cols) +{ + struct row **rows = calloc((size_t)n, sizeof(*rows)); + if (!rows) abort(); + for (int i = 0; i < n; i++) rows[i] = row_new(cols); + return rows; +} + +/* Copy a row into a new width. Cells past the width drop. */ +static struct row *row_resized(const struct row *r, int old_cols, int cols) +{ + struct row *n = row_new(cols); + int keep = old_cols < cols ? old_cols : cols; + memcpy(n->c, r->c, (size_t)keep * sizeof(struct cell)); + n->len = r->len < cols ? r->len : cols; + n->wrapped = r->wrapped; + return n; +} + +struct reflow { + int cols, rows; + struct row **out; + int count, cap; + struct row *cur; + int x; + bool cursor_set, saved_set, prompt_set; + int cursor_x, cursor_y, saved_x, saved_y, prompt; + bool pending; +}; + +static void reflow_end_row(struct reflow *r, bool wrapped) +{ + if (!r->cur) r->cur = row_new(r->cols); + r->cur->len = r->x > r->cols ? r->cols : r->x; + r->cur->wrapped = wrapped; + if (r->count == r->cap) { + r->cap = r->cap ? r->cap * 2 : 64; + r->out = realloc(r->out, (size_t)r->cap * sizeof(*r->out)); + if (!r->out) abort(); + } + r->out[r->count++] = r->cur; + r->cur = NULL; + r->x = 0; +} + +static struct cell *reflow_dst(struct reflow *r) +{ + if (!r->cur) r->cur = row_new(r->cols); + return r->cur->c; +} + +/* + * Append one logical line. Pads drop and wide glyphs wrap again. A cursor + * keeps its cell. A cursor on a pad moves to the next glyph. + */ +static void reflow_append(struct reflow *r, const struct cell *cells, int len, + int cursor, int saved) +{ + for (int i = 0; i < len;) { + const struct cell *c = &cells[i]; + if (c->pad || c->width == 0) { + int at = c->pad ? r->x : r->x - 1; + if (cursor == i) { + r->cursor_x = at < 0 ? 0 : at; + r->cursor_y = r->count; + r->cursor_set = true; + r->pending = false; + } + if (saved == i) { + r->saved_x = at < 0 ? 0 : at; + r->saved_y = r->count; + r->saved_set = true; + } + i++; + continue; + } + int n = c->width == 2 && r->cols > 1 ? 2 : 1; + if (r->x + n > r->cols) { + if (r->x < r->cols) { + reflow_dst(r)[r->x] = default_blank(); + reflow_dst(r)[r->x].pad = true; + r->x++; + } + reflow_end_row(r, true); + } + if (cursor == i) { + r->cursor_x = r->x; + r->cursor_y = r->count; + r->cursor_set = true; + r->pending = false; + } + if (saved == i) { + r->saved_x = r->x; + r->saved_y = r->count; + r->saved_set = true; + } + struct cell *dst = reflow_dst(r); + dst[r->x] = *c; + if (n == 2) { + dst[r->x + 1] = *c; + dst[r->x + 1].n = 0; + dst[r->x + 1].width = 0; + memset(dst[r->x + 1].cp, 0, sizeof(c->cp)); + } + r->x += n; + i++; + } + if (cursor == len) { + r->cursor_y = r->count; + r->cursor_x = r->x == r->cols ? r->cols - 1 : r->x; + r->pending = r->x == r->cols; + r->cursor_set = true; + } + if (saved == len) { + r->saved_y = r->count; + r->saved_x = r->x == r->cols ? r->cols - 1 : r->x; + r->saved_set = true; + } +} + +static void hist_reflow_tail(struct model *m) +{ + if (m->hist_len <= 0) + return; + int start = m->hist_len - 1; + while (start > 0 && m->hist[start - 1]->wrapped) + start--; + + bool line_open = m->hist[m->hist_len - 1]->wrapped; + struct reflow rf = { .cols = m->cols, .rows = m->rows }; + for (int i = start; i < m->hist_len; i++) { + const struct row *src = m->hist[i]; + int len = src->wrapped && src->len < m->cols ? m->cols : src->len; + reflow_append(&rf, src->c, len, -1, -1); + } + if (rf.x || rf.count == 0) + reflow_end_row(&rf, false); + free(rf.cur); + + if (line_open) { + struct row *last = rf.out[rf.count - 1]; + last->wrapped = true; + if (last->len < m->cols) { + for (int x = last->len; x < m->cols; x++) { + last->c[x] = default_blank(); + last->c[x].pad = true; + } + last->len = m->cols; + } + } + + for (int i = start; i < m->hist_len; i++) + free(m->hist[i]); + m->hist_len = start; + while (m->hist_len + rf.count > m->hist_cap) { + m->hist_cap = m->hist_cap ? m->hist_cap * 2 : 64; + m->hist = realloc(m->hist, (size_t)m->hist_cap * sizeof(*m->hist)); + if (!m->hist) abort(); + } + for (int i = 0; i < rf.count; i++) + m->hist[m->hist_len++] = rf.out[i]; + free(rf.out); +} + +/* + * Reflow the normal screen and its history to a new width. Logical lines + * wrap again. Lines at or after the prompt row (OSC 133;A) are cut, not + * wrapped. Blank rows below the cursor drop. Rows more than one screen + * below the cursor drop. + */ +static void reflow(struct model *m, int cols, int rows) +{ + int old_cols = m->cols, old_rows = m->rows; + int active = old_rows; + while (active > m->y + 1 && active > m->saved.y + 1 && + !m->scr[active - 1]->len && !m->scr[active - 1]->wrapped) + active--; + struct reflow r = { .cols = cols, .rows = rows }; + int total_in = m->hist_len + active; + struct cell *line = malloc(((size_t)old_cols + (size_t)m->saved.x + 2) * sizeof(*line)); + if (!line) abort(); + for (int i = 0; i < total_in; i++) { + int y = i - m->hist_len; + const struct row *src = i < m->hist_len ? m->hist[i] : m->scr[y]; + int len = src->len; + bool wrapped = src->wrapped; + if (m->prompt >= 0 && y >= m->prompt) { + if (r.x) reflow_end_row(&r, false); + if (y == m->prompt) { + r.prompt = r.count; + r.prompt_set = true; + } + if (y == m->y) { + r.cursor_y = r.count; + r.cursor_x = m->x < cols ? m->x : cols - 1; + r.pending = false; + r.cursor_set = true; + } + if (y == m->saved.y) { + r.saved_y = r.count; + r.saved_x = m->saved.x < cols ? m->saved.x : cols - 1; + r.saved_set = true; + } + int keep = len > cols ? cols : len; + memcpy(reflow_dst(&r), src->c, (size_t)keep * sizeof(struct cell)); + r.x = keep; + reflow_end_row(&r, wrapped); + continue; + } + if (wrapped && len < old_cols) len = old_cols; + if (y == m->y && len < m->x + m->pending) len = m->x + m->pending; + if (y == m->saved.y && len <= m->saved.x) len = m->saved.x + 1; + for (int x = 0; x < len; x++) + line[x] = x < old_cols ? src->c[x] : default_blank(); + reflow_append(&r, line, len, y == m->y ? m->x + m->pending : -1, + y == m->saved.y ? m->saved.x : -1); + if (!wrapped || i + 1 == total_in) reflow_end_row(&r, false); + } + free(line); + if (r.x) reflow_end_row(&r, false); + free(r.cur); + + int visible = rows - (cols > old_cols ? old_rows - active : 0); + if (visible < 1) visible = 1; + int start = r.count > visible ? r.count - visible : 0; + if (start > r.cursor_y) start = r.cursor_y; + int history = start; + if (KNOWN_HISTORY_COUNTS_ROWS) { + if (history > m->limit) history = m->limit; + } else { + int lines = 0; + int first = start; + for (int i = start - 1; i >= 0; i--) { + if (i == start - 1 || !r.out[i]->wrapped) { + if (lines == m->limit) { + first = i + 1; + break; + } + lines++; + } + if (i == 0) first = 0; + } + history = start - first; + } + + m->cols = cols; + term_hist_clear(m); + for (int i = start - history; i < start; i++) { + hist_push(m, r.out[i]); + r.out[i] = NULL; + } + free_rows(m->scr, old_rows); + m->scr = calloc((size_t)rows, sizeof(*m->scr)); + if (!m->scr) abort(); + for (int y = 0; y < rows; y++) { + if (start + y < r.count) { + m->scr[y] = r.out[start + y]; + r.out[start + y] = NULL; + } else { + m->scr[y] = row_new(cols); + } + } + for (int i = 0; i < r.count; i++) free(r.out[i]); + free(r.out); + + m->rows = rows; + m->x = r.cursor_x < cols ? r.cursor_x : cols - 1; + m->y = r.cursor_y - start; + m->pending = r.pending && m->autowrap; + if (m->prompt >= 0) { + if (!r.prompt_set || r.prompt - start >= rows) m->prompt = -1; + else m->prompt = r.prompt > start ? r.prompt - start : 0; + } + /* A saved cursor on a row below the new screen moves to the last row + * and keeps its column. */ + if (r.saved_set && r.saved_y - start < rows) { + m->saved.x = r.saved_x < cols ? r.saved_x : cols - 1; + m->saved.y = r.saved_y < start ? 0 : r.saved_y - start; + } else { + m->saved.y = rows - 1; + } + if (m->saved.x >= cols) m->saved.x = cols - 1; + if (m->saved.y >= rows) m->saved.y = rows - 1; +} + +/* Resize without reflow: the alternate screen, or the same width. */ +static void resize_plain(struct model *m, int cols, int rows) +{ + int old_cols = m->cols, old_rows = m->rows; + if (cols != old_cols) { + for (int y = 0; y < old_rows; y++) { + struct row *n = row_resized(m->scr[y], old_cols, cols); + free(m->scr[y]); + m->scr[y] = n; + } + if (KNOWN_ALT_RESIZE_CUTS_NORMAL && m->alternate) { + for (int y = 0; y < m->normal_rows; y++) + if (m->normal[y]->len > cols) row_cut(m->normal[y], cols, m->normal_cols); + for (int i = 0; i < m->hist_len; i++) + if (m->hist[i]->len > cols) row_cut(m->hist[i], cols, m->normal_cols); + } + m->cols = cols; + } + if (rows < old_rows && m->y >= rows) { + int shift = m->y - rows + 1; + for (int i = 0; i < shift; i++) scroll_up_full(m); + m->y -= shift; + m->saved.y = m->saved.y >= shift ? m->saved.y - shift : 0; + } + if (rows != old_rows) { + /* Growth pulls rows back from history above the old rows. New + * blank rows take the pen colors. */ + int pull = 0; + if (rows > old_rows && !m->alternate) + pull = m->hist_len < rows - old_rows ? m->hist_len : rows - old_rows; + struct row **scr = calloc((size_t)rows, sizeof(*scr)); + if (!scr) abort(); + int y = 0; + for (int i = 0; i < pull; i++) scr[y++] = m->hist[m->hist_len - pull + i]; + for (int i = 0; i < old_rows; i++) { + if (y < rows) scr[y++] = m->scr[i]; + else free(m->scr[i]); + } + int first_new = y; + while (y < rows) scr[y++] = row_new(cols); + free(m->scr); + m->scr = scr; + m->rows = rows; + for (y = first_new; y < rows; y++) erase_row(m, y); + m->hist_len -= pull; + m->hist_lines = 0; + for (int i = 0; i < m->hist_len; i++) + if (i == 0 || !m->hist[i - 1]->wrapped) + m->hist_lines++; + m->y += pull; + m->saved.y += pull; + if (m->prompt >= 0) m->prompt += pull; + } + /* A shrink that cuts the prompt row off the normal screen ends the + * prompt. */ + if (!m->alternate && m->prompt >= rows) m->prompt = -1; + if (cols != old_cols) m->pending = false; + if (m->x >= cols) m->x = cols - 1; + if (m->y >= rows) m->y = rows - 1; + if (m->saved.x >= cols) m->saved.x = cols - 1; + if (m->saved.y >= rows) m->saved.y = rows - 1; +} + +static bool resize(struct model *m, int cols, int rows) +{ + if (cols > TERM_MAX_COLS) cols = TERM_MAX_COLS; + if (cols == m->cols && rows == m->rows) return true; + if (cols > MAX_TABS) return false; + m->top = 0; + m->bottom = rows - 1; + if (cols != m->cols && !m->alternate) + reflow(m, cols, rows); + else + resize_plain(m, cols, rows); + return true; +} + +static void set_alternate(struct model *m, bool on) +{ + if (m->alternate == on) return; + if (on) { + m->normal = m->scr; + m->normal_cols = m->cols; + m->normal_rows = m->rows; + m->normal_cursor = save_cursor(m); + m->normal_saved = m->saved; + m->normal_visible = m->visible; + m->normal_style = m->style; + m->normal_blink = m->blink; + m->scr = rows_new(m->rows, m->cols); + m->alternate = true; + m->x = m->y = 0; + m->pending = false; + } else { + int cols = m->cols, rows = m->rows; + free_rows(m->scr, m->rows); + m->scr = m->normal; + m->normal = NULL; + m->cols = m->normal_cols; + m->rows = m->normal_rows; + m->alternate = false; + m->saved = m->normal_saved; + m->visible = m->normal_visible; + m->style = m->normal_style; + m->blink = m->normal_blink; + restore_cursor(m, &m->normal_cursor); + m->x = m->normal_cursor.x; + m->y = m->normal_cursor.y; + m->pending = m->normal_cursor.pending; + resize(m, cols, rows); + } + m->top = 0; + m->bottom = m->rows - 1; +} + +/* ---- modes and resets ---- */ + +static void soft_reset(struct model *m) +{ + m->app_cursor = m->app_keypad = m->backarrow = false; + m->insert = m->origin = m->reverse = m->lnm = false; + m->focus = m->sync = false; + m->mouse = m->mouse_enc = 0; + m->bracketed = false; + m->autowrap = m->visible = true; + m->blink = true; + m->style = TERM_CURSOR_BLOCK; + m->top = 0; + m->bottom = m->rows - 1; + m->decsace = 0; + memset(m->keyboard, 0, sizeof(m->keyboard)); + pen_reset(&m->pen); +} + +static void full_reset(struct model *m) +{ + set_alternate(m, false); + soft_reset(m); + m->x = m->y = 0; + m->pending = false; + m->saved = initial_saved(); + m->last_n = 0; + m->g[0] = m->g[1] = 'B'; + m->gl = 0; + m->prompt = -1; + tabs_default(m); + for (int y = 0; y < m->rows; y++) erase_row(m, y); +} + +static int mode_get(const struct model *m, int mode, bool priv) +{ + if (!priv) { + if (mode == 4) return m->insert; + if (mode == 20) return m->lnm; + return -1; + } + switch (mode) { + case 1: return m->app_cursor; + case 12: return m->blink; + case 25: return m->visible; + case 67: return m->backarrow; + case 1004: return m->focus; + case 2004: return m->bracketed; + case 2026: return m->sync; + case 5: return m->reverse; + case 6: return m->origin; + case 7: return m->autowrap; + case 47: case 1047: case 1049: return m->alternate; + case 69: return 0; + case 9: case 1000: case 1001: case 1002: case 1003: return m->mouse == mode; + case 1005: case 1006: case 1015: case 1016: return m->mouse_enc == mode; + default: return -1; + } +} + +static void mode_set(struct model *m, int mode, bool on, bool priv) +{ + if (!priv) { + if (mode == 4) m->insert = on; + else if (mode == 20) m->lnm = on; + return; + } + switch (mode) { + case 1: m->app_cursor = on; break; + case 12: m->blink = on; break; + case 25: m->visible = on; break; + case 67: m->backarrow = on; break; + case 1004: m->focus = on; break; + case 2004: m->bracketed = on; break; + case 2026: m->sync = on; break; + case 5: m->reverse = on; break; + case 6: + m->origin = on; + move_to(m, 0, on ? m->top : 0); + break; + case 7: + m->autowrap = on; + m->pending = false; + break; + case 1048: + if (on) m->saved = save_cursor(m); + else restore_cursor(m, &m->saved); + break; + case 47: case 1047: case 1049: + set_alternate(m, on); + break; + case 9: case 1000: case 1001: case 1002: case 1003: + m->mouse = on ? mode : 0; + break; + case 1005: case 1006: case 1015: case 1016: + m->mouse_enc = on ? mode : 0; + break; + default: + break; + } +} + +/* ---- CSI ---- */ + +static int count(const struct model *m) +{ + return m->param[0] > 0 ? m->param[0] : 1; +} + +static bool csi_allowed(const struct model *m, unsigned char final) +{ + char prefix = m->prefix, inter = m->inter; + if (final == 'm') return !prefix && !inter; + if (final == 'H' || final == 'f') return !prefix && !inter && !m->sub; + if (inter) { + bool plain = (final == 'q' && inter == ' ') || (final == 'p' && inter == '!') || + (final == 'x' && inter == '*'); + if (plain) { + if (prefix || m->sub) return false; + } else if (final == 'p' && inter == '$') { + if (m->sub) return false; + } else if (final == 'r' && inter == '$') { + if (prefix) return false; + } else { + return false; + } + } + if (prefix && + final != 'u' && + !(final == 'c' && (prefix == '>' || prefix == '=')) && + !(final == 'q' && prefix == '>' && !inter) && + !(final == 'p' && inter == '$' && prefix == '?') && + !(final == 'W' && prefix == '?') && + !((final == 'h' || final == 'l') && prefix == '?')) + return false; + if (m->sub && !(inter == '$' && final == 'r')) return false; + return true; +} + +static void keyboard_csi(struct model *m) +{ + if (m->nparam > (m->prefix == '=' ? 2 : 1)) return; + struct keyboard *k = &m->keyboard[m->alternate]; + unsigned flags = (unsigned)m->param[0] & 31; + if (m->prefix == '?') { + reply(m, "\033[?%uu", k->flags); + } else if (m->prefix == '>') { + if (k->depth == 32) { + memmove(k->saved, k->saved + 1, sizeof(k->saved) - sizeof(k->saved[0])); + k->depth--; + } + k->saved[k->depth++] = k->flags; + k->flags = flags; + } else if (m->prefix == '<') { + unsigned n = (unsigned)count(m); + if (n > k->depth) { + k->depth = 0; + k->flags = 0; + } else { + k->depth -= n; + k->flags = k->saved[k->depth]; + } + } else if (m->prefix == '=') { + int mode = m->nparam > 1 && m->param[1] ? m->param[1] : 1; + if (mode == 1) k->flags = flags; + else if (mode == 2) k->flags |= flags; + else if (mode == 3) k->flags &= ~flags; + } +} + +static void csi_dispatch(struct model *m, unsigned char final) +{ + if (!csi_allowed(m, final)) return; + int n = count(m); + switch (final) { + case 'm': sgr(m); break; + case 'H': case 'f': { + long y = (long)(m->param[0] > 0 ? m->param[0] - 1 : 0) + top_limit(m); + if (y > bottom_limit(m)) y = bottom_limit(m); + long x = m->nparam > 1 && m->param[1] > 0 ? m->param[1] - 1 : 0; + move_to(m, (int)(x > m->cols - 1 ? m->cols - 1 : x), (int)y); + break; + } + case 'r': { + if (m->inter == '$') { + deccara(m); + break; + } + int top = n - 1; + int bottom = m->nparam > 1 && m->param[1] ? m->param[1] - 1 : m->rows - 1; + if (top < bottom && bottom < m->rows) { + m->top = top; + m->bottom = bottom; + move_to(m, 0, m->origin ? top : 0); + } + break; + } + case 'L': case 'M': + if (m->y < m->top || m->y > m->bottom) break; + scroll_region(m, m->y, m->bottom, n, final == 'L', false); + move_to(m, 0, m->y); + break; + case 'S': case 'T': + if (m->nparam > 1) break; + m->pending = false; + scroll_region(m, m->top, m->bottom, n, final == 'T', true); + break; + case 'd': { + long y = (long)n - 1 + top_limit(m); + move_to(m, m->x, (int)(y > bottom_limit(m) ? bottom_limit(m) : y)); + break; + } + case 'b': { + int times = n > m->cols * m->rows ? m->cols * m->rows : n; + for (int i = 0; i < times && m->last_n; i++) + put_glyph(m, m->last, m->last_n, m->pen.attrs); + break; + } + case 'A': move_v(m, -n, false); break; + case 'B': case 'e': move_v(m, n, false); break; + case 'E': move_v(m, n, true); break; + case 'F': move_v(m, -n, true); break; + case 'a': case 'C': move_h(m, n); break; + case 'D': move_h(m, -n); break; + case 'G': case '`': { + long x = (long)n - 1; + move_to(m, (int)(x > m->cols - 1 ? m->cols - 1 : x), m->y); + break; + } + case 's': + if (!m->prefix) m->saved = save_cursor(m); + break; + case 'u': + if (!m->prefix) restore_cursor(m, &m->saved); + else keyboard_csi(m); + break; + case 'I': case 'Z': { + int times = n > m->cols ? m->cols : n, dir = final == 'I' ? 1 : -1; + while (times-- > 0) { + int x = m->x + dir; + while (x > 0 && x < m->cols - 1 && !m->tabs[x]) x += dir; + m->x = x < 0 ? 0 : x >= m->cols ? m->cols - 1 : x; + } + m->pending = false; + break; + } + case 'g': + if (m->param[0] == 0) m->tabs[m->x] = false; + else if (m->param[0] == 3) memset(m->tabs, 0, sizeof(m->tabs)); + break; + case 'W': + if (m->prefix == '?' && m->param[0] == 5) tabs_default(m); + break; + case 'n': + if (m->prefix) break; + if (m->param[0] == 5) reply(m, "\033[0n"); + else if (m->param[0] == 6) reply(m, "\033[%d;%dR", m->y - top_limit(m) + 1, m->x + 1); + break; + case 'J': m->pending = false; erase_display(m, m->param[0]); break; + case 'K': m->pending = false; erase_line(m, m->param[0]); break; + case 'X': { + m->pending = false; + int k = n > m->cols - m->x ? m->cols - m->x : n; + erase_cells(m, m->y, m->x, m->x + k); + break; + } + case 'x': + if (m->inter == '*') m->decsace = m->param[0]; + break; + case 'P': m->pending = false; delete_chars(m, n); break; + case '@': m->pending = false; insert_chars(m, n); break; + case 'c': + if (m->prefix == '>') reply(m, "\033[>0;10;0c"); + else if (!m->prefix && m->param[0] == 0) reply(m, "\033[?62;22c"); + break; + case 't': + if (!m->prefix && m->param[0] == 14) + reply(m, "\033[4;%d;%dt", m->rows * 20, m->cols * 10); + break; + case 'h': case 'l': + for (int i = 0; i < m->nparam; i++) + mode_set(m, m->param[i], final == 'h', m->prefix == '?'); + break; + case 'q': + if (m->prefix == '>' && !m->inter && (!m->nparam || !m->param[0])) { + reply(m, "\033P>|wsap_term\033\\"); + } else if (!m->prefix && m->inter == ' ') { + int ps = m->nparam ? m->param[0] : 0; + if (ps == 0) { + m->style = TERM_CURSOR_BLOCK; + m->blink = true; + } else if (ps <= 6) { + m->style = ps < 3 ? TERM_CURSOR_BLOCK : ps < 5 ? TERM_CURSOR_UNDERLINE + : TERM_CURSOR_BAR; + m->blink = ps & 1; + } + } + break; + case 'p': + if (m->inter == '!') { + soft_reset(m); + move_to(m, 0, 0); + } else if (m->inter == '$' && m->nparam >= 1) { + int set = mode_get(m, m->param[0], m->prefix == '?'); + reply(m, "\033[%s%d;%d$y", m->prefix == '?' ? "?" : "", m->param[0], + set < 0 ? 0 : set ? 1 : 2); + } + break; + default: + break; + } +} + +/* ---- strings ---- */ + +static void osc_finish(struct model *m) +{ + const char *s = m->str; + size_t len = m->str_len; + if (!m->alternate && len >= 5 && !strncmp(s, "133;", 4) && + (len == 5 || s[5] == ';')) { + if (s[4] == 'A') m->prompt = m->y; + if (s[4] == 'C' || s[4] == 'D') m->prompt = -1; + } +} + +static void dcs_finish(struct model *m) +{ + const char *s = m->str; + size_t len = m->str_len; + if (len < 2 || s[0] != '$' || s[1] != 'q') return; + const char *spec = s + 2; + size_t n = len - 2; + char body[256]; + if (n == 2 && !memcmp(spec, " q", 2)) { + int shape = m->style == TERM_CURSOR_BLOCK ? (m->blink ? 1 : 2) + : m->style == TERM_CURSOR_UNDERLINE ? (m->blink ? 3 : 4) + : (m->blink ? 5 : 6); + reply(m, "\033P1$r%d q\033\\", shape); + } else if (n == 1 && spec[0] == 'm') { + sgr_report(m, body, sizeof(body)); + reply(m, "\033P1$r%sm\033\\", body); + } else if (n == 1 && spec[0] == 'r') { + reply(m, "\033P1$r%d;%dr\033\\", m->top + 1, m->bottom + 1); + } else if (n == 1 && spec[0] == 's') { + reply(m, "\033P1$r%d;%ds\033\\", 1, m->cols); + } else if (n == 1 && spec[0] == 't') { + reply(m, "\033P1$r%dt\033\\", m->rows); + } else if (n == 2 && !memcmp(spec, "\"p", 2)) { + reply(m, "\033P1$r64;1\"p\033\\"); + } else if (n == 2 && !memcmp(spec, "\"q", 2)) { + reply(m, "\033P1$r0\"q\033\\"); + } else if (n == 2 && !memcmp(spec, "$|", 2)) { + reply(m, "\033P1$r%d$|\033\\", m->cols); + } else if (n == 2 && !memcmp(spec, "$}", 2)) { + reply(m, "\033P1$r0$}\033\\"); + } else if (n == 2 && !memcmp(spec, "$~", 2)) { + reply(m, "\033P1$r0$~\033\\"); + } else if (n == 2 && !memcmp(spec, "*x", 2)) { + reply(m, "\033P1$r%d*x\033\\", m->decsace); + } else if (n == 2 && !memcmp(spec, "*|", 2)) { + reply(m, "\033P1$r%d*|\033\\", m->rows); + } else { + reply(m, "\033P0$r\033\\"); + } +} + +static void str_add(struct model *m, unsigned char c) +{ + if (m->str_len < STR_KEEP) m->str[m->str_len] = (char)c; + m->str_len++; +} + +/* A string longer than STR_KEEP keeps its true length, so it matches no + * short request. */ +static void str_finish(struct model *m) +{ + if (m->str_kind == OSC) osc_finish(m); + else if (m->str_kind == DCS) dcs_finish(m); + m->str_len = 0; +} + +/* ---- the byte parser ---- */ + +static void esc_dispatch(struct model *m, unsigned char ch) +{ + m->state = GROUND; + switch (ch) { + case '=': m->app_keypad = true; break; + case '>': m->app_keypad = false; break; + case 'H': m->tabs[m->x] = true; break; + case 'c': full_reset(m); break; + case '[': + m->state = CSI; + m->nparam = 0; + m->param[0] = 0; + m->sub = 0; + m->prefix = m->inter = 0; + m->csi_phase = 0; + break; + case ']': m->state = STR; m->str_kind = OSC; m->str_len = 0; break; + case 'P': m->state = STR; m->str_kind = DCS; m->str_len = 0; break; + case '_': case '^': case 'X': m->state = STR; m->str_kind = IGNORE; m->str_len = 0; break; + case '#': m->state = HASH; break; + case '%': m->state = DOCS; break; + case '(': case ')': case '*': case '+': case '-': case '.': case '/': + m->slot = (char)ch; + m->state = CHARSET; + break; + case 'D': case 'E': + m->pending = false; + if (ch == 'E') m->x = 0; + line_feed(m); + break; + case 'M': + m->pending = false; + reverse_index(m); + break; + case '7': m->saved = save_cursor(m); break; + case '8': restore_cursor(m, &m->saved); break; + default: break; + } +} + +static void utf8_interrupt(struct model *m) +{ + if (m->need) { + m->need = 0; + print_cp(m, 0xFFFD); + } +} + +static void control(struct model *m, unsigned char c) +{ + switch (c) { + case 0x1b: m->state = ESC; break; + case '\n': case 0x0b: case 0x0c: + m->pending = false; + if (m->lnm || (m->flags & TERM_FEED_ONLCR)) m->x = 0; + line_feed(m); + break; + case '\r': m->pending = false; m->x = 0; break; + case '\b': + m->pending = false; + if (m->x > 0) m->x--; + break; + case '\t': { + m->pending = false; + int x = m->x + 1; + while (x < m->cols - 1 && !m->tabs[x]) x++; + m->x = x < m->cols ? x : m->cols - 1; + break; + } + case 0x0e: m->gl = 1; break; + case 0x0f: m->gl = 0; break; + default: break; + } +} + +/* Ground-state bytes: printable ASCII, C0 controls, DEL, and UTF-8. Each + * maximal invalid UTF-8 subpart prints one U+FFFD. */ +static void ground(struct model *m, unsigned char c) +{ + if (c < 0x20 || c == 0x7f) { + utf8_interrupt(m); + if (c < 0x20) control(m, c); + return; + } + if (m->need) { + if (c >= m->lo && c <= m->hi) { + m->acc = m->acc << 6 | (c & 0x3fu); + m->lo = 0x80; + m->hi = 0xbf; + if (--m->need == 0) print_cp(m, m->acc); + return; + } + utf8_interrupt(m); + } + if (c < 0x80) { + print_cp(m, c); + return; + } + m->lo = 0x80; + m->hi = 0xbf; + if (c >= 0xc2 && c <= 0xdf) { m->need = 1; m->acc = c & 0x1fu; } + else if (c == 0xe0) { m->need = 2; m->acc = c & 0x0fu; m->lo = 0xa0; } + else if (c == 0xed) { m->need = 2; m->acc = c & 0x0fu; m->hi = 0x9f; } + else if (c >= 0xe1 && c <= 0xef) { m->need = 2; m->acc = c & 0x0fu; } + else if (c == 0xf0) { m->need = 3; m->acc = c & 0x07u; m->lo = 0x90; } + else if (c == 0xf4) { m->need = 3; m->acc = c & 0x07u; m->hi = 0x8f; } + else if (c >= 0xf1 && c <= 0xf3) { m->need = 3; m->acc = c & 0x07u; } + else print_cp(m, 0xFFFD); +} + +static void csi_byte(struct model *m, unsigned char c) +{ + if (c == 0x7f) return; + if (c >= 0x40) { + if (m->csi_phase != 3) csi_dispatch(m, c); + m->state = GROUND; + return; + } + if (m->csi_phase == 3) return; + if (c >= '0' && c <= ';') { + if (m->csi_phase == 2) { + m->csi_phase = 3; + return; + } + m->csi_phase = 1; + if (!m->nparam) m->nparam = 1; + int *p = &m->param[m->nparam - 1]; + if (c <= '9') { + *p = *p < 214748364 ? *p * 10 + (c - '0') : 0x7fffffff; + } else { + if (m->nparam == 16) { + m->csi_phase = 3; + return; + } + if (c == ':') m->sub |= (uint16_t)(1u << m->nparam); + m->param[m->nparam++] = 0; + } + } else if (c >= '<') { + if (m->csi_phase != 0 || m->prefix) m->csi_phase = 3; + else m->prefix = (char)c; + } else { + if (m->csi_phase == 2) { + m->csi_phase = 3; + } else { + m->csi_phase = 2; + m->inter = (char)c; + } + } +} + +static void feed_byte(struct model *m, unsigned char c) +{ + /* A byte of 0x80 or more cancels any sequence except a string. */ + if (c >= 0x80 && m->state != GROUND && m->state != STR) { + m->state = GROUND; + ground(m, c); + return; + } + switch (m->state) { + case GROUND: + ground(m, c); + break; + case ESC: + if (c < 0x20) { + m->state = GROUND; + ground(m, c); + } else { + esc_dispatch(m, c); + } + break; + case CSI: + /* ESC restarts, CAN and SUB cancel, other C0 controls run inside + * the sequence (ECMA-48 5.5, xterm). */ + if (c == 0x1b) m->state = ESC; + else if (c == 0x18 || c == 0x1a) m->state = GROUND; + else if (c < 0x20) control(m, c); + else csi_byte(m, c); + break; + case CHARSET: + if (m->slot == '(' || m->slot == ')') m->g[m->slot == ')'] = (char)c; + m->state = GROUND; + break; + case HASH: + m->state = GROUND; + if (c == '8') { + for (int y = 0; y < m->rows; y++) { + struct row *r = m->scr[y]; + for (int x = 0; x < m->cols; x++) { + r->c[x] = default_blank(); + r->c[x].cp[0] = 'E'; + } + r->len = m->cols; + r->wrapped = false; + } + move_to(m, 0, 0); + } + break; + case DOCS: + m->state = GROUND; + if (c != 'G' && c != '@' && c != '/') { + ground(m, '%'); + ground(m, c); + } + break; + case STR: + if (m->str_kind == IGNORE) { + if (c == 0x07 || c == 0x18 || c == 0x1a) m->state = GROUND; + else if (c == 0x1b) m->state = STR_ESC; + break; + } + if (c == 0x07) { + str_finish(m); + m->state = GROUND; + } else if (c == 0x1b) { + m->state = STR_ESC; + } else if (c < 0x20) { + m->str_len = 0; + m->state = GROUND; + ground(m, c); + } else { + str_add(m, c); + } + break; + case STR_ESC: + if (m->str_kind == IGNORE) { + if (c == '\\' || c == 0x07 || c == 0x18 || c == 0x1a) { + m->state = GROUND; + } else { + m->state = ESC; + feed_byte(m, c); + } + break; + } + if (c == '\\') { + str_finish(m); + m->state = GROUND; + } else { + m->str_len = 0; + m->state = ESC; + feed_byte(m, c); + } + break; + } +} + +/* ---- public ---- */ + +struct model *model_new(struct term_vector2i size, int history) +{ + struct model *m = calloc(1, sizeof(*m)); + if (!m) abort(); + m->cols = size.x < 1 ? 80 : size.x; + m->rows = size.y < 1 ? 24 : size.y; + m->limit = history_limit(history); + m->scr = rows_new(m->rows, m->cols); + m->autowrap = m->visible = m->blink = true; + m->style = TERM_CURSOR_BLOCK; + m->bottom = m->rows - 1; + m->g[0] = m->g[1] = 'B'; + m->saved = initial_saved(); + m->prompt = -1; + pen_reset(&m->pen); + tabs_default(m); + (void)KNOWN_HISTORY_COUNTS_ROWS; + return m; +} + +void model_free(struct model *m) +{ + if (!m) return; + free_rows(m->scr, m->rows); + free_rows(m->normal, m->normal_rows); + term_hist_clear(m); + free(m->hist); + free(m->reply); + for (int i = 0; i < m->out_cap; i++) free(m->out[i]); + free(m->out); + free(m); +} + +void model_feed(struct model *m, const char *s, size_t len, unsigned flags) +{ + m->flags = flags; + for (size_t i = 0; i < len; i++) feed_byte(m, (unsigned char)s[i]); +} + +bool model_resize(struct model *m, struct term_vector2i size) +{ + if (size.x < 1 || size.y < 1) return false; + return resize(m, size.x, size.y); +} + +struct term_vector2i model_size(const struct model *m) +{ + return (struct term_vector2i){ .x = m->cols, .y = m->rows }; +} + +int model_history(const struct model *m) +{ + return m->alternate ? 0 : m->hist_len; +} + +const struct model_cell *model_row(const struct model *m, int row) +{ + int hist = model_history(m); + if (row < 0 || (row >= hist && row - hist >= m->rows)) return NULL; + + struct model *mm = (struct model *)m; + const struct row *r = row < hist ? m->hist[row] : m->scr[row - hist]; + if (mm->out_cols != m->cols) { + for (int i = 0; i < mm->out_cap; i++) { + free(mm->out[i]); + mm->out[i] = NULL; + } + mm->out_cols = m->cols; + } + if (row >= mm->out_cap) { + int cap = mm->out_cap ? mm->out_cap : 16; + while (cap <= row) { + if (cap > INT_MAX / 2) abort(); + cap *= 2; + } + struct model_cell **out = realloc(mm->out, (size_t)cap * sizeof(*out)); + if (!out) abort(); + memset(out + mm->out_cap, 0, (size_t)(cap - mm->out_cap) * sizeof(*out)); + mm->out = out; + mm->out_cap = cap; + } + if (!mm->out[row]) { + mm->out[row] = calloc((size_t)m->cols, sizeof(*mm->out[row])); + if (!mm->out[row]) abort(); + } + for (int x = 0; x < m->cols; x++) { + const struct cell *c = &r->c[x]; + struct model_cell *o = &mm->out[row][x]; + memcpy(o->cp, c->cp, sizeof(o->cp)); + o->n = c->n; + o->width = c->width; + o->attrs = c->attrs; + o->fg = c->fg; + o->bg = c->bg; + } + return mm->out[row]; +} + +struct term_cursor model_cursor(const struct model *m) +{ + return (struct term_cursor){ + .pos = { .x = m->x, .y = m->y }, + .style = (enum term_cursor_style)m->style, + .visible = m->visible, + .blink = m->blink, + }; +} + +struct term_modes model_modes(const struct model *m) +{ + return (struct term_modes){ + .keyboard = m->keyboard[m->alternate].flags, + .mouse = (enum term_mouse)m->mouse, + .mouse_encoding = (enum term_mouse_encoding)m->mouse_enc, + .app_cursor = m->app_cursor, + .app_keypad = m->app_keypad, + .backarrow_bs = m->backarrow, + .bracketed_paste = m->bracketed, + .focus_events = m->focus, + .alternate = m->alternate, + .reverse_video = m->reverse, + .synchronized = m->sync, + }; +} + +const char *model_replies(const struct model *m, size_t *len) +{ + *len = m->reply_len; + return m->reply ? m->reply : ""; +} + +void model_take_replies(struct model *m) +{ + m->reply_len = 0; +} diff --git a/tests/model.h b/tests/model.h new file mode 100644 index 0000000..9a4316f --- /dev/null +++ b/tests/model.h @@ -0,0 +1,50 @@ +#ifndef WSAP_TERM_MODEL_H +#define WSAP_TERM_MODEL_H + +/* + * A slow reference model of the terminal. It defines correct behavior for + * the input that tests/term/fuzz_gen.h makes. model.c names its sources: + * xterm and ECMA-48 for controls, Unicode for UTF-8, and this terminal's + * own design where xterm has no rule (reflow, history, clusters). + * + * The model reads nothing from the terminal. Tests compare the two through + * the public API only. + */ + +#include "term/term.h" +#include "term/unicode/grapheme.h" + +#include <stdbool.h> +#include <stddef.h> +#include <stdint.h> + +struct model_cell { + uint32_t cp[GRAPHEME_MAX_LEN]; + uint8_t n; /* code points; 0 for the right half of a wide cell */ + uint8_t width; /* 1, 2 for a wide cell, 0 for its right half */ + uint16_t attrs; /* TERM_ATTR_* */ + term_color fg, bg; +}; + +struct model; + +/* history is the history limit, as in term_options.scrollback. */ +struct model *model_new(struct term_vector2i size, int history); +void model_free(struct model *m); +void model_feed(struct model *m, const char *s, size_t len, unsigned flags); +bool model_resize(struct model *m, struct term_vector2i size); + +struct term_vector2i model_size(const struct model *m); +/* History rows. The alternate screen shows no history, so it reports 0. */ +int model_history(const struct model *m); +/* Row 0 is the oldest history row. Row model_history() is the top screen + * row. Return NULL for an invalid row. Different rows can be read together. + * A feed or resize ends the lifetime of returned row data. */ +const struct model_cell *model_row(const struct model *m, int row); +struct term_cursor model_cursor(const struct model *m); +struct term_modes model_modes(const struct model *m); +/* Replies since the last model_take_replies(). */ +const char *model_replies(const struct model *m, size_t *len); +void model_take_replies(struct model *m); + +#endif diff --git a/tests/model_row_test.c b/tests/model_row_test.c new file mode 100644 index 0000000..7afca05 --- /dev/null +++ b/tests/model_row_test.c @@ -0,0 +1,29 @@ +#include "model.h" + +#include <assert.h> +#include <string.h> + +int main(void) +{ + struct model *m = model_new((struct term_vector2i){ .x = 10, .y = 3 }, 10); + model_feed(m, "A\r\nB\r\nC\r\nD", strlen("A\r\nB\r\nC\r\nD"), 0); + assert(model_history(m) == 1); + assert(model_row(m, -1) == NULL); + assert(model_row(m, 4) == NULL); + + const struct model_cell *oldest = model_row(m, 0); + const struct model_cell *next = model_row(m, 1); + const struct model_cell *last = model_row(m, 3); + assert(oldest[0].cp[0] == 'A'); + assert(next[0].cp[0] == 'B'); + assert(last[0].cp[0] == 'D'); + assert(oldest != next && next != last); + + model_feed(m, "E", 1, 0); + assert(model_row(m, 3)[1].cp[0] == 'E'); + assert(model_resize(m, (struct term_vector2i){ .x = 12, .y = 3 })); + assert(model_row(m, 3)[0].cp[0] == 'D'); + assert(model_row(m, 3)[1].cp[0] == 'E'); + model_free(m); + return 0; +} diff --git a/tests/model_test.c b/tests/model_test.c new file mode 100644 index 0000000..395097e --- /dev/null +++ b/tests/model_test.c @@ -0,0 +1,458 @@ +/* + * Compare the terminal with the reference model (model.c) through the + * public API: every history page and screen row through a snapshot, the + * cursor, the modes, the history size, and the reply bytes. + * + * Each seed makes a random geometry and history limit, then runs four + * steps of generated input. The terminal gets each step in random chunks, + * so the fast paths and the byte parser both run. One step in four ends + * with a random resize. Every step ends with queries (DSR, DECRQSS, + * DECRQM), so hidden state such as the pen, the margins, and the modes + * shows in the replies. + * + * On a mismatch the test shrinks the input, prints a short C string that + * reproduces it, and fails. + * + * Usage: model-test [seeds] [first-seed] + */ +#include "model.h" +#include "fuzz_gen.h" +#include "term/term.h" + +#include <stdio.h> +#include <limits.h> +#include <stdlib.h> +#include <string.h> + +/* Queries that make hidden state visible. They change nothing. */ +static const char probe[] = + "\033[6n\033P$qm\033\\\033P$qr\033\\\033P$q q\033\\" + "\033[?6$p\033[?7$p\033[4$p\033[20$p"; + +struct event { + char *bytes; /* NULL for a resize */ + size_t len; + unsigned flags; + struct term_vector2i size; +}; + +struct script { + struct term_vector2i size; + int history; + struct event *ev; + int n, cap; +}; + +static void script_add(struct script *s, struct event e) +{ + if (s->n == s->cap) { + s->cap = s->cap ? s->cap * 2 : 16; + s->ev = realloc(s->ev, (size_t)s->cap * sizeof(*s->ev)); + if (!s->ev) abort(); + } + s->ev[s->n++] = e; +} + +static void script_free(struct script *s) +{ + for (int i = 0; i < s->n; i++) free(s->ev[i].bytes); + free(s->ev); + *s = (struct script){0}; +} + +static char *dup_bytes(const char *p, size_t n) +{ + char *d = malloc(n ? n : 1); + if (!d) abort(); + memcpy(d, p, n); + return d; +} + +struct replies { + char *s; + size_t len, cap; +}; + +static void on_reply(void *ud, const char *bytes, size_t len) +{ + struct replies *r = ud; + if (r->len + len > r->cap) { + r->cap = (r->len + len) * 2; + r->s = realloc(r->s, r->cap); + if (!r->s) abort(); + } + memcpy(r->s + r->len, bytes, len); + r->len += len; +} + +static void dump(FILE *f, const char *s, size_t n) +{ + fputc('"', f); + for (size_t i = 0; i < n; i++) { + unsigned char c = (unsigned char)s[i]; + if (c == 27) fputs("\\033", f); + else if (c == '\r') fputs("\\r", f); + else if (c == '\n') fputs("\\n", f); + else if (c == '\t') fputs("\\t", f); + else if (c == '"' || c == '\\') fprintf(f, "\\%c", c); + else if (c >= 32 && c < 127) fputc(c, f); + else fprintf(f, "\\x%02x\"\"", c); + } + fputc('"', f); +} + +static char glyph(uint32_t cp) +{ + return cp >= 32 && cp < 127 ? (char)cp : cp ? '?' : '.'; +} + +/* Print every retained row of both, side by side. */ +static void dump_all(struct model *m, struct term *t, struct term_snapshot *snap, FILE *log) +{ + int total = term_scrollback(t).total, mtotal = model_history(m); + struct term_vector2i size = term_size(t), msize = model_size(m); + int rows = total + size.y > mtotal + msize.y ? total + size.y : mtotal + msize.y; + fprintf(log, " term (history %d)%*s model (history %d)\n", total, size.x > 8 ? size.x - 8 : 0, + "", mtotal); + for (int r = 0; r < rows; r++) { + fprintf(log, " %3d |", r); + if (r < total + size.y) { + int offset = r < total ? total - r : 0; + term_scroll_to(t, offset); + term_snapshot_update(snap, t); + const struct term_cell *row = term_snapshot_row(snap, r < total ? 0 : r - total); + for (int x = 0; x < size.x; x++) { + uint32_t cp[GRAPHEME_MAX_LEN] = {0}; + if (row[x].text) term_snapshot_cluster(snap, row[x].text, cp, GRAPHEME_MAX_LEN); + fputc(glyph(cp[0]), log); + } + } else { + fprintf(log, "%*s", size.x, ""); + } + fprintf(log, "| |"); + if (r < mtotal + msize.y) { + const struct model_cell *row = model_row(m, r); + for (int x = 0; x < msize.x; x++) fputc(glyph(row[x].n ? row[x].cp[0] : 0), log); + } + fprintf(log, "|\n"); + } + term_scroll_to(t, 0); + struct term_cursor c = term_cursor(t), mc = model_cursor(m); + fprintf(log, " cursor term %d,%d model %d,%d\n", c.pos.x, c.pos.y, mc.pos.x, mc.pos.y); +} + +/* Returns NULL when the terminal matches the model, else a description. + * Details go to log when it is not NULL. */ +static const char *compare(struct model *m, struct term *t, struct term_snapshot *snap, + struct replies *got, char *why, size_t cap, FILE *log) +{ + size_t want_len; + const char *want = model_replies(m, &want_len); + if (want_len != got->len || (want_len && memcmp(want, got->s, want_len))) { + if (log) { + fprintf(log, "replies: term "); + dump(log, got->s ? got->s : "", got->len); + fprintf(log, "\n model "); + dump(log, want, want_len); + fputc('\n', log); + } + snprintf(why, cap, "replies differ"); + return why; + } + struct term_vector2i size = term_size(t), msize = model_size(m); + if (size.x != msize.x || size.y != msize.y) { + snprintf(why, cap, "size %dx%d, model %dx%d", size.x, size.y, msize.x, msize.y); + return why; + } + struct term_cursor c = term_cursor(t), mc = model_cursor(m); + if (c.pos.x != mc.pos.x || c.pos.y != mc.pos.y || c.style != mc.style || + c.visible != mc.visible || c.blink != mc.blink) { + snprintf(why, cap, "cursor %d,%d style %d visible %d blink %d; model %d,%d %d %d %d", + c.pos.x, c.pos.y, c.style, c.visible, c.blink, + mc.pos.x, mc.pos.y, mc.style, mc.visible, mc.blink); + return why; + } + struct term_modes md = term_modes(t), mm = model_modes(m); + if (md.keyboard != mm.keyboard || md.mouse != mm.mouse || + md.mouse_encoding != mm.mouse_encoding || md.app_cursor != mm.app_cursor || + md.app_keypad != mm.app_keypad || md.backarrow_bs != mm.backarrow_bs || + md.bracketed_paste != mm.bracketed_paste || md.focus_events != mm.focus_events || + md.alternate != mm.alternate || md.reverse_video != mm.reverse_video || + md.synchronized != mm.synchronized) { + snprintf(why, cap, "modes differ (alternate %d/%d reverse %d/%d keypad %d/%d)", + md.alternate, mm.alternate, md.reverse_video, mm.reverse_video, + md.app_keypad, mm.app_keypad); + return why; + } + /* Visit each retained row. Wide-line totals can start as estimates. */ + int total = model_history(m); + term_scroll_to(t, 0); + term_scroll(t, INT_MAX); + for (int offset = total;; offset = offset > size.y ? offset - size.y : 0) { + if (!offset) + term_scroll_to(t, 0); + if (!term_snapshot_update(snap, t)) { + snprintf(why, cap, "snapshot update failed"); + return why; + } + for (int y = 0; y < size.y; y++) { + const struct term_cell *row = term_snapshot_row(snap, y); + int retained = total - offset + y; + const struct model_cell *want_row = model_row(m, retained); + for (int x = 0; x < size.x; x++) { + const struct model_cell *e = &want_row[x]; + uint32_t cps[GRAPHEME_MAX_LEN] = {0}; + int n = row[x].text ? term_snapshot_cluster(snap, row[x].text, cps, + GRAPHEME_MAX_LEN) : 0; + if (n == e->n && row[x].width == e->width && row[x].attrs == e->attrs && + row[x].fg == e->fg && row[x].bg == e->bg && + !memcmp(cps, e->cp, (size_t)n * sizeof(*cps))) + continue; + if (log) { + fprintf(log, "retained row %d (offset %d):\n term: ", retained, offset); + for (int k = 0; k < size.x; k++) { + uint32_t cp[GRAPHEME_MAX_LEN] = {0}; + if (row[k].text) + term_snapshot_cluster(snap, row[k].text, cp, GRAPHEME_MAX_LEN); + fputc(glyph(cp[0]), log); + } + fprintf(log, "\n model: "); + want_row = model_row(m, retained); + for (int k = 0; k < size.x; k++) + fputc(glyph(want_row[k].n ? want_row[k].cp[0] : 0), log); + fputc('\n', log); + e = &want_row[x]; + } + snprintf(why, cap, + "cell %d,%d: term U+%04X (%d cps) w%u attrs %#x fg %#x bg %#x; " + "model U+%04X (%u cps) w%u attrs %#x fg %#x bg %#x", + x, y, cps[0], n, row[x].width, row[x].attrs, row[x].fg, row[x].bg, + e->cp[0], e->n, e->width, e->attrs, e->fg, e->bg); + term_scroll_to(t, 0); + return why; + } + } + if (!offset) break; + term_scroll(t, -(offset > size.y ? size.y : offset)); + } + int observed_total = term_scrollback(t).total; + if (observed_total != total) { + snprintf(why, cap, "visited history %d rows, model %d", observed_total, total); + return why; + } + term_scroll_to(t, 0); + return NULL; +} + +enum chunking { CHUNK_RANDOM, CHUNK_WHOLE, CHUNK_BYTES }; + +/* Run a script. Returns the event index of the first mismatch, or -1. */ +static int run(const struct script *s, enum chunking chunks, uint64_t chunk_seed, + char *why, size_t cap, bool verbose) +{ + struct replies got = {0}; + struct term_options opt = { + .size = s->size, .scrollback = s->history, + .callbacks = { .ud = &got, .reply = on_reply }, + }; + struct term *t = term_new(&opt); + struct model *m = model_new(s->size, s->history); + struct term_snapshot *snap = term_snapshot_new(); + if (!t || !snap) abort(); + uint64_t saved_rng = rng; + rng = chunk_seed | 1; + int bad = -1; + for (int i = 0; i < s->n && bad < 0; i++) { + const struct event *e = &s->ev[i]; + if (e->bytes) { + model_feed(m, e->bytes, e->len, e->flags); + for (size_t k = 0; k < e->len;) { + size_t n = chunks == CHUNK_WHOLE ? e->len - k + : chunks == CHUNK_BYTES ? 1 + : 1 + (size_t)rn((int)(e->len - k)); + term_feed(t, e->bytes + k, n, e->flags); + k += n; + } + } else { + bool a = model_resize(m, e->size); + bool b = term_resize(t, e->size); + if (a != b) { + snprintf(why, cap, "resize result differs"); + bad = i; + break; + } + } + if (compare(m, t, snap, &got, why, cap, verbose ? stderr : NULL)) { + bad = i; + if (verbose) dump_all(m, t, snap, stderr); + } + model_take_replies(m); + got.len = 0; + } + rng = saved_rng; + free(got.s); + term_snapshot_free(snap); + model_free(m); + term_free(t); + return bad; +} + +static bool fails(const struct script *s, char *why, size_t cap) +{ + return run(s, CHUNK_WHOLE, 1, why, cap, false) >= 0 || + run(s, CHUNK_BYTES, 1, why, cap, false) >= 0; +} + +/* Remove byte ranges and events while the script still fails. */ +static void shrink(struct script *s) +{ + char why[512]; + for (int i = s->n - 1; i >= 0; i--) { + struct event e = s->ev[i]; + memmove(s->ev + i, s->ev + i + 1, (size_t)(s->n - i - 1) * sizeof(*s->ev)); + s->n--; + if (fails(s, why, sizeof(why))) { + free(e.bytes); + continue; + } + memmove(s->ev + i + 1, s->ev + i, (size_t)(s->n - i) * sizeof(*s->ev)); + s->ev[i] = e; + s->n++; + } + for (int i = 0; i < s->n; i++) { + struct event *e = &s->ev[i]; + if (!e->bytes) continue; + for (size_t step = e->len / 2 ? e->len / 2 : 1; step >= 1; step /= 2) { + for (size_t at = 0; at + step <= e->len;) { + char *keep = dup_bytes(e->bytes, e->len); + size_t keep_len = e->len; + memmove(e->bytes + at, e->bytes + at + step, e->len - at - step); + e->len -= step; + if (fails(s, why, sizeof(why))) { + free(keep); + continue; + } + free(e->bytes); + e->bytes = keep; + e->len = keep_len; + at++; + } + if (step == 1) break; + } + } +} + +static void report(struct script *s, int seed, const char *why) +{ + fprintf(stderr, "seed %d: %s\n", seed, why); + shrink(s); + char buf2[512]; + fprintf(stderr, "shrunk: %dx%d history %d\n", s->size.x, s->size.y, s->history); + for (int i = 0; i < s->n; i++) { + if (s->ev[i].bytes) { + fprintf(stderr, " feed%s ", s->ev[i].flags ? " (ONLCR)" : ""); + dump(stderr, s->ev[i].bytes, s->ev[i].len); + fputc('\n', stderr); + } else { + fprintf(stderr, " resize %dx%d\n", s->ev[i].size.x, s->ev[i].size.y); + } + } + int at = run(s, CHUNK_WHOLE, 1, buf2, sizeof(buf2), true); + if (at < 0) at = run(s, CHUNK_BYTES, 1, buf2, sizeof(buf2), true); + fprintf(stderr, "shrunk mismatch after event %d: %s\n", at, buf2); +} + +/* Fixed cases that the generator reaches rarely. */ +static int fixed_cases(void) +{ + static const struct { + int cols, rows, history; + const char *input; + } cases[] = { + { 12, 3, 5, "a中e\xcc\x81日本 \033[1;3;4;9;38;5;123;48;2;1;2;3mQ\033[0mR" }, + { 8, 4, 5, "\033[2;3r\033[?6hZ\nQ\nR\033[?6l\033[r" }, + { 12, 3, 50, "one two three four five\r\nsix seven eight\r\nnine" }, + { 10, 3, 50, "\033]133;A\007$ ls -la /usr/share\r\nout1\r\n\033]133;A\007$ " }, + { 6, 3, 5, "\xf0\x9f\x87\xa9\xf0\x9f\x87\xaa\xe2\x9d\xa4\xef\xb8\x8f" "ab\xe2\x98\xba\xef\xb8\x8e" }, + { 8, 3, 5, "\033(0lqqk\033(Bx\x0e" "abc\x0f" "abc" }, + { 8, 3, 5, "abcdefgh\033[3;4H\033[31;1m\0337\033[0m\033[H\0338X" }, + { 6, 2, 5, "abcdef\xe3\x80z\xed\xa0\x80\x80" }, + }; + static const struct term_vector2i sizes[] = { + { .x = 5, .y = 3 }, { .x = 16, .y = 4 }, { .x = 7, .y = 3 }, { .x = 20, .y = 5 }, + { .x = 1, .y = 2 }, { .x = 3, .y = 6 }, + }; + for (size_t i = 0; i < sizeof(cases) / sizeof(cases[0]); i++) { + struct script s = { + .size = { .x = cases[i].cols, .y = cases[i].rows }, + .history = cases[i].history, + }; + script_add(&s, (struct event){ + .bytes = dup_bytes(cases[i].input, strlen(cases[i].input)), + .len = strlen(cases[i].input) }); + script_add(&s, (struct event){ .bytes = dup_bytes(probe, sizeof(probe) - 1), + .len = sizeof(probe) - 1 }); + for (size_t k = 0; k < sizeof(sizes) / sizeof(sizes[0]); k++) + script_add(&s, (struct event){ .size = sizes[k] }); + char why[512]; + int at = run(&s, CHUNK_BYTES, 1, why, sizeof(why), true); + if (at >= 0) { + fprintf(stderr, "fixed case %zu event %d: %s\n", i, at, why); + report(&s, -(int)i, why); + script_free(&s); + return 1; + } + script_free(&s); + } + return 0; +} + +int main(int argc, char **argv) +{ + int seeds = argc > 1 ? atoi(argv[1]) : 20000; + int first = argc > 2 ? atoi(argv[2]) : 1; + if (fixed_cases()) return 1; + static const int limits[] = { 0, 5, 50, 4096 }; + for (int seed = first; seed < first + seeds; seed++) { + rng = 0x9e3779b97f4a7c15ull * (uint64_t)seed; + struct script s = { + .size = { .x = 4 + rn(37), .y = 2 + rn(11) }, + .history = limits[rn(4)], + }; + struct term_vector2i size = s.size; + for (int step = 0; step < 4; step++) { + blen = 0; + gen(10 + rn(30), size.x, size.y); + put(probe); + unsigned flags = rn(5) == 0 ? TERM_FEED_ONLCR : 0; + script_add(&s, (struct event){ .bytes = dup_bytes(buf, blen), .len = blen, + .flags = flags }); + if (rn(4) == 0) { + size = (struct term_vector2i){ .x = 1 + rn(44), .y = 1 + rn(14) }; + script_add(&s, (struct event){ .size = size }); + } + } + if (getenv("MODEL_TEST_PRINT")) { + fprintf(stderr, "seed %d: %dx%d history %d\n", seed, s.size.x, s.size.y, s.history); + for (int i = 0; i < s.n; i++) { + if (!s.ev[i].bytes) { + fprintf(stderr, " resize %dx%d\n", s.ev[i].size.x, s.ev[i].size.y); + continue; + } + fprintf(stderr, " feed "); + dump(stderr, s.ev[i].bytes, s.ev[i].len); + fputc('\n', stderr); + } + } + char why[512]; + uint64_t chunk_seed = rng; + if (run(&s, CHUNK_RANDOM, chunk_seed, why, sizeof(why), false) >= 0) { + run(&s, CHUNK_RANDOM, chunk_seed, why, sizeof(why), true); + report(&s, seed, why); + script_free(&s); + return 1; + } + script_free(&s); + } + return 0; +} diff --git a/tests/osc_test.c b/tests/osc_test.c new file mode 100644 index 0000000..ea0ef08 --- /dev/null +++ b/tests/osc_test.c @@ -0,0 +1,400 @@ +#include "term_api.h" +#include "term/unicode/unicode.h" +#include <assert.h> +#include <stdbool.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> + +static void write_seq(struct term *t, const char *s) +{ + for (; *s; s++) tt_write(t, s, 1); +} + +static void write_buf(struct term *t, const char *s, size_t len) +{ + tt_write(t, s, len); +} + +static char reply[256]; +static void capture_reply(void *ud, const char *s, size_t len) +{ + (void)ud; + snprintf(reply, sizeof(reply), "%.*s", (int)len, s); +} + +static char title[256]; +static void capture_title(void *ud, const char *s) +{ + (void)ud; + snprintf(title, sizeof(title), "%s", s ? s : ""); +} + +static char clip_sel[16]; +static char clip_data[4096]; +static size_t clip_len; +static int clip_count; +static void capture_clip(void *ud, const char *sel, const char *data, size_t len) +{ + (void)ud; + snprintf(clip_sel, sizeof(clip_sel), "%s", sel ? sel : ""); + assert(len < sizeof(clip_data)); + memcpy(clip_data, data, len); + clip_len = len; + clip_count++; +} + +static char notify_title[256]; +static char notify_body[1024]; +static int notify_count; +static void capture_notify(void *ud, const char *t_, const char *b) +{ + (void)ud; + snprintf(notify_title, sizeof(notify_title), "%s", t_ ? t_ : ""); + snprintf(notify_body, sizeof(notify_body), "%s", b ? b : ""); + notify_count++; +} + +static struct term *osc_test_term(void) +{ + struct term *t = tt_new_opts(80, 24, &(struct term_callbacks){ + .reply = capture_reply, + .title = capture_title, + .clipboard = capture_clip, + .notify = capture_notify, + }); + assert(t); + return t; +} + +/* Check the OSC 8 link or the detected URL at a viewport cell. NULL means + * no link. */ +static void assert_link(struct term *t, int col, int row, const char *want) +{ + char *url = tt_link_at(t, col, row); + if (want) + assert(url && !strcmp(url, want)); + else + assert(!url); + free(url); +} + +static uint32_t cursor_color(struct term *t) +{ + return term_snapshot_view(tt_snapshot_for(t))->palette[TERM_PALETTE_CURSOR]; +} + +/* Send a color query and check the reply. */ +static void assert_query(struct term *t, const char *query, const char *want) +{ + reply[0] = 0; + write_seq(t, query); + if (strcmp(reply, want)) { + fprintf(stderr, "reply to %s: got \"%s\" want \"%s\"\n", query, reply, want); + assert(!"reply mismatch"); + } +} + +static void reset_captures(void) +{ + reply[0] = 0; + title[0] = clip_sel[0] = clip_data[0] = notify_title[0] = notify_body[0] = 0; + clip_len = 0; + clip_count = 0; + notify_count = 0; +} + +static void test_osc52(void) +{ + struct term *t = osc_test_term(); + reset_captures(); + write_seq(t, "\033]52;c;aGVsbG8=\007"); + assert(clip_count == 1 && !strcmp(clip_sel, "c")); + assert(clip_len == 5 && !memcmp(clip_data, "hello", 5)); + + reset_captures(); + write_seq(t, "\033]52;c;aGVsbG8=\033\\"); + assert(clip_count == 1 && clip_len == 5 && !memcmp(clip_data, "hello", 5)); + + reset_captures(); + static const char bulk[] = "\033]52;c;aGVsbG8=\007"; + write_buf(t, bulk, sizeof(bulk) - 1); + assert(clip_count == 1 && clip_len == 5 && !memcmp(clip_data, "hello", 5)); + + reset_captures(); + char big[7 + 400 + 1]; + memcpy(big, "\033]52;c;", 7); + for (int i = 0; i < 400; i += 4) + memcpy(big + 7 + i, "QUJD", 4); + big[407] = '\007'; + write_buf(t, big, sizeof(big)); + assert(clip_count == 1 && clip_len == 300); + for (int i = 0; i < 300; i += 3) + assert(!memcmp(clip_data + i, "ABC", 3)); + + reset_captures(); + write_seq(t, "\033]52;c;\007"); + assert(clip_count == 1 && clip_len == 0); + + reset_captures(); + write_seq(t, "\033]52;c;?\007"); + write_seq(t, "\033]52;c;!!!invalid!!!\007"); + write_seq(t, "\033]52;aGVsbG8=\007"); + write_seq(t, "\033]52;;aGVsbG8=\007"); + assert(clip_count == 0); + + write_seq(t, "\033]52;pc;aGVsbG8=\007"); + assert(clip_count == 1 && !strcmp(clip_sel, "pc")); + assert(clip_len == 5 && !memcmp(clip_data, "hello", 5)); + + reset_captures(); + write_seq(t, "\033]52;s;AAH9\007"); + assert(clip_count == 1 && !strcmp(clip_sel, "s") && clip_len == 3); + assert(clip_data[0] == 0 && clip_data[1] == 1 && clip_data[2] == (char)0xfd); + term_free(t); +} + +/* An SGR reset must not drop an open link. + * + * The fast SGR path caches the cell flags, and those flags carry the link id. + * The reset used to set the cached value to zero, which cleared the link from + * every cell written after it. Write the whole sequence at once so the fast + * path runs. */ +static void test_sgr_reset_keeps_link(void) +{ + struct term *t = osc_test_term(); + reset_captures(); + + static const char seq[] = "\033]8;;https://reset.dev\033\\\033[0mX"; + write_buf(t, seq, sizeof(seq) - 1); + assert(tt_cell(t, 0, 0)->attrs & TERM_ATTR_LINK); + assert_link(t, 0, 0, "https://reset.dev"); + + /* The same reset in its own write, which takes the normal parser. */ + static const char seq2[] = "\033]8;;https://second.dev\033\\\033[0m"; + write_buf(t, seq2, sizeof(seq2) - 1); + write_buf(t, "Y", 1); + assert(tt_cell(t, 1, 0)->attrs & TERM_ATTR_LINK); + assert_link(t, 1, 0, "https://second.dev"); + + term_free(t); +} + +static void test_osc8(void) +{ + struct term *t = osc_test_term(); + reset_captures(); + + write_seq(t, "\033]8;;https://example.com\033\\link text\033]8;;\033\\after"); + for (int c = 0; c < 9; c++) { + assert(tt_cell(t, c, 0)->attrs & TERM_ATTR_LINK); + assert_link(t, c, 0, "https://example.com"); + } + for (int c = 9; c < 14; c++) { + assert(!(tt_cell(t, c, 0)->attrs & TERM_ATTR_LINK)); + assert_link(t, c, 0, NULL); + } + + /* Each cell keeps the URI of its own link. */ + write_seq(t, "\033[2;1H\033]8;;https://other.org\033\\bbb\033]8;;\033\\"); + write_seq(t, "\033[3;1H\033]8;;https://example.com\033\\ccc\033]8;;\033\\"); + assert(tt_cell(t, 0, 1)->attrs & TERM_ATTR_LINK); + assert(tt_cell(t, 0, 2)->attrs & TERM_ATTR_LINK); + assert_link(t, 0, 1, "https://other.org"); + assert_link(t, 2, 1, "https://other.org"); + assert_link(t, 0, 2, "https://example.com"); + assert_link(t, 0, 0, "https://example.com"); + + /* An erase removes the link. */ + write_seq(t, "\033[4;1H\033]8;;https://gone.io\033\\zzz\033]8;;\033\\"); + assert(tt_cell(t, 0, 3)->attrs & TERM_ATTR_LINK); + write_seq(t, "\033[4;1H\033[K"); + assert(!(tt_cell(t, 0, 3)->attrs & TERM_ATTR_LINK)); + assert_link(t, 0, 3, NULL); + + write_seq(t, "\033[5;1H\033]8;x\033\\nop\033]8;xyz\033\\qq"); + assert(!(tt_cell(t, 0, 4)->attrs & TERM_ATTR_LINK)); + assert_link(t, 0, 4, NULL); + assert(!(tt_cell(t, 4, 4)->attrs & TERM_ATTR_LINK)); + write_seq(t, "\033]8;;https://stay.net\033\\\033]8;bogus\033\\ST\033]8;;\033\\"); + assert(tt_cell(t, 5, 4)->attrs & TERM_ATTR_LINK); + assert_link(t, 5, 4, "https://stay.net"); + + write_seq(t, "\033[6;1H\033]8;id=grp;https://params.dev\007p1\033]8;;\007"); + assert(tt_cell(t, 0, 5)->attrs & TERM_ATTR_LINK); + assert_link(t, 0, 5, "https://params.dev"); + + /* A link stays on its cells when they scroll into the history. */ + for (int i = 0; i < 30; i++) + write_buf(t, "filler\r\n", 8); + assert(tt_history(t) > 0); + term_scroll(t, tt_history(t)); + assert(tt_view_cell(t, 0, 0)->attrs & TERM_ATTR_LINK); + assert_link(t, 0, 0, "https://example.com"); + term_scroll_to(t, 0); + + write_seq(t, "\033[7;1H"); + for (int i = 0; i < 8300; i++) { + char seq[64]; + int n = snprintf(seq, sizeof(seq), + "\033]8;;https://h/%d\033\\A\010", i); + write_buf(t, seq, (size_t)n); + } + assert_link(t, 0, 6, "https://h/8299"); + assert_link(t, -1, 0, NULL); + assert_link(t, 0, -1, NULL); + assert_link(t, 80, 0, NULL); + assert_link(t, 0, 24, NULL); + + write_seq(t, "\033[?1049h"); + write_seq(t, "\033[1;1H\033]8;;https://altscreen.test\033\\ALT\033]8;;\033\\"); + assert_link(t, 0, 0, "https://altscreen.test"); + write_seq(t, "\033[?1049l"); + term_scroll(t, tt_history(t)); + assert_link(t, 0, 0, "https://example.com"); + term_scroll_to(t, 0); + write_seq(t, "\033[8;1HSee file:///home/user/code/file.c#L10-L20 for info"); + assert_link(t, 10, 7, "file:///home/user/code/file.c#L10-L20"); + + term_free(t); +} + +static bool text_is(size_t (*get)(const struct term *, char *, size_t), + const struct term *t, const char *want) +{ + char buf[256]; + return get(t, buf, sizeof(buf)) == strlen(want) && !strcmp(buf, want); +} + +static void test_osc_misc(void) +{ + struct term *t = osc_test_term(); + reset_captures(); + + write_seq(t, "\033]7;file://host/home/user/proj\007"); + assert(text_is(term_cwd, t, "/home/user/proj")); + + write_seq(t, "\033]7;file:///a%20b\033\\"); + assert(text_is(term_cwd, t, "/a b")); + + const char *bulk = "\033]7;file://h/bulk%20dir\007"; + write_buf(t, bulk, strlen(bulk)); + assert(text_is(term_cwd, t, "/bulk dir")); + + write_seq(t, "\033]7;https://x/y\007"); + write_seq(t, "\033]7;notauri\007"); + assert(text_is(term_cwd, t, "/bulk dir")); + + write_seq(t, "\033]9;hello there\033\\"); + assert(notify_count == 1); + assert(!strcmp(notify_title, "wsap_term")); + assert(!strcmp(notify_body, "hello there")); + write_seq(t, "\033]9;4;3;50\007"); + assert(notify_count == 1); + + write_seq(t, "\033]777;notify;Title;Body text\007"); + assert(notify_count == 2); + assert(!strcmp(notify_title, "Title")); + assert(!strcmp(notify_body, "Body text")); + write_seq(t, "\033]777;other;T;B\007"); + assert(notify_count == 2); + + write_seq(t, "\033]22;pointer\033\\"); + assert(text_is(term_pointer_shape, t, "pointer")); + write_seq(t, "\033]22;text\007"); + assert(text_is(term_pointer_shape, t, "text")); + write_seq(t, "\033]22;\033\\"); + assert(!term_pointer_shape(t, NULL, 0)); + + term_free(t); +} + +/* OSC 4, 10, 11, and 12 change the colors of one terminal. The reset codes + * restore the colors of its theme. */ +static void test_osc_colors(void) +{ + struct term_theme theme; + term_theme_default(&theme); + struct term *t = osc_test_term(); + struct term *other = osc_test_term(); + reset_captures(); + + /* tt_new starts with the built-in theme. */ + assert(tt_color(t, TERM_COLOR_FG) == 0xFFFFFFFFu && theme.fg == 0xFFFFFFFFu); + assert(tt_color(t, TERM_COLOR_BG) == 0xFF1E1E2Eu && theme.bg == 0xFF1E1E2Eu); + assert(cursor_color(t) == 0xFF89B4FAu && theme.cursor == 0xFF89B4FAu); + assert_query(t, "\033]10;?\007", "\033]10;rgb:ffff/ffff/ffff\033\\"); + + write_seq(t, "\033]10;#ff0000\007"); + assert(tt_color(t, TERM_COLOR_FG) == 0xFFFF0000u); + assert_query(t, "\033]10;?\007", "\033]10;rgb:ffff/0000/0000\033\\"); + write_seq(t, "\033]110\007"); + assert(tt_color(t, TERM_COLOR_FG) == theme.fg); + assert_query(t, "\033]10;?\033\\", "\033]10;rgb:ffff/ffff/ffff\033\\"); + + write_seq(t, "\033]11;#00ff00\033\\"); + assert(tt_color(t, TERM_COLOR_BG) == 0xFF00FF00u); + assert_query(t, "\033]11;?\007", "\033]11;rgb:0000/ffff/0000\033\\"); + write_seq(t, "\033]111\007"); + assert(tt_color(t, TERM_COLOR_BG) == theme.bg); + assert_query(t, "\033]11;?\007", "\033]11;rgb:1e1e/1e1e/2e2e\033\\"); + + write_seq(t, "\033]12;#0000ff\007"); + assert(cursor_color(t) == 0xFF0000FFu); + assert_query(t, "\033]12;?\007", "\033]12;rgb:0000/0000/ffff\033\\"); + write_seq(t, "\033]112\033\\"); + assert(cursor_color(t) == theme.cursor); + assert_query(t, "\033]12;?\007", "\033]12;rgb:8989/b4b4/fafa\033\\"); + + uint32_t ansi5 = tt_color(t, term_color_index(5)); + assert(ansi5 == theme.ansi[5]); + write_seq(t, "\033]4;5;#ff0000\007"); + assert(tt_color(t, term_color_index(5)) == 0xFFFF0000u); + assert(tt_color(t, term_color_index(4)) == theme.ansi[4]); + + /* A color change in one terminal does not change another terminal. */ + write_seq(t, "\033]10;#123456\007"); + assert(tt_color(other, term_color_index(5)) == theme.ansi[5]); + assert(tt_color(other, TERM_COLOR_FG) == theme.fg); + assert_query(other, "\033]10;?\007", "\033]10;rgb:ffff/ffff/ffff\033\\"); + assert(tt_color(t, TERM_COLOR_FG) == 0xFF123456u); + + write_seq(t, "\033]104;5\007"); + assert(tt_color(t, term_color_index(5)) == ansi5); + write_seq(t, "\033]4;1;#010203;2;#040506\007"); + assert(tt_color(t, term_color_index(1)) == 0xFF010203u); + assert(tt_color(t, term_color_index(2)) == 0xFF040506u); + write_seq(t, "\033]104\007"); + assert(tt_color(t, term_color_index(1)) == theme.ansi[1]); + assert(tt_color(t, term_color_index(2)) == theme.ansi[2]); + + term_free(other); + term_free(t); +} + +static void test_osc_legacy_titles(void) +{ + struct term *t = osc_test_term(); + reset_captures(); + + write_seq(t, "\033]legacy title\007"); + assert(!strcmp(title, "legacy title")); + write_seq(t, "\033]\007"); + assert(!strcmp(title, "")); + write_seq(t, "\033]2adjacent\033\\"); + assert(!strcmp(title, "adjacent")); + + term_free(t); +} + +int main(void) +{ + test_osc52(); + test_osc8(); + test_sgr_reset_keeps_link(); + test_osc_misc(); + test_osc_colors(); + test_osc_legacy_titles(); + puts("PASS: OSC tests"); + return 0; +} diff --git a/tests/resize_failure_test.c b/tests/resize_failure_test.c new file mode 100644 index 0000000..5d7e63c --- /dev/null +++ b/tests/resize_failure_test.c @@ -0,0 +1,152 @@ +#define _GNU_SOURCE +#include "term_api.h" + +#include <assert.h> +#include <dlfcn.h> +#include <stdbool.h> +#include <stddef.h> +#include <stdio.h> +#include <stdlib.h> + +#ifndef __has_feature +#define __has_feature(x) 0 +#endif + +#if defined(__SANITIZE_ADDRESS__) || defined(__SANITIZE_THREAD__) +#define SANITIZED 1 +#elif __has_feature(address_sanitizer) || __has_feature(memory_sanitizer) || \ + __has_feature(thread_sanitizer) +#define SANITIZED 1 +#endif + +#ifndef SANITIZED +static bool fail_next_calloc; +static bool fail_reallocs; + +void *calloc(size_t n, size_t size) +{ + static void *(*real_calloc)(size_t, size_t); + if (!real_calloc) + real_calloc = (void *(*)(size_t, size_t))dlsym(RTLD_NEXT, "calloc"); + if (fail_next_calloc) { + fail_next_calloc = false; + return NULL; + } + return real_calloc(n, size); +} + +void *realloc(void *p, size_t size) +{ + static void *(*real_realloc)(void *, size_t); + if (!real_realloc) + real_realloc = (void *(*)(void *, size_t))dlsym(RTLD_NEXT, "realloc"); + if (fail_reallocs) + return NULL; + return real_realloc(p, size); +} +#endif + +int main(void) +{ +#ifdef SANITIZED + puts("resize failure test skipped under sanitizer"); + return 0; +#else + struct term *term = tt_new(10, 2); + assert(term); + tt_write(term, "abc", 3); + + fail_next_calloc = true; + bool resized = tt_resize(term, 20, 2); + /* The resize must reach the failing allocation. */ + assert(!fail_next_calloc); + + /* A failed resize must leave the old terminal geometry usable as a unit. */ + assert(!resized); + assert(tt_cols(term) == 10); + assert(tt_rows(term) == 2); + assert(tt_cell(term, 0, 0)->text == 'a'); + assert(tt_cell(term, 2, 0)->text == 'c'); + + tt_write(term, "\r12345678901", 12); + assert(tt_cursor_x(term) == 1); + assert(tt_cursor_y(term) == 1); + assert(tt_cell(term, 0, 1)->text == '1'); + + /* The failed lazy allocation must not prevent a later resize retry. */ + resized = tt_resize(term, 20, 2); + assert(resized); + assert(tt_cols(term) == 20); + assert(tt_rows(term) == 2); + assert(tt_cell(term, 0, 0)->text == '1'); + term_free(term); + + term = tt_new(10, 3); + assert(term); + const char margins[] = "\033[?69h\033[1;5s\033[H"; + tt_write(term, margins, sizeof(margins) - 1); + fail_next_calloc = true; + resized = tt_resize(term, 20, 3); + assert(!fail_next_calloc); + + /* A failed resize keeps the margins that fit the old width. */ + assert(!resized); + assert(tt_cols(term) == 10); + tt_write(term, "abcdefg", 7); + assert(tt_cell(term, 4, 0)->text == 'e'); + assert(tt_cell(term, 5, 0)->text != 'f'); + assert(tt_cell(term, 0, 1)->text == 'f'); + term_free(term); + + term = tt_new(10, 2); + assert(term); + tt_write(term, "normal", 6); + fail_next_calloc = true; + tt_write(term, "\033[?1049h", 8); + assert(!fail_next_calloc); + + /* A failed alternate-screen allocation keeps the normal screen active. */ + assert(!term_modes(term).alternate); + assert(tt_cell(term, 0, 0)->text == 'n'); + assert(tt_cell(term, 5, 0)->text == 'l'); + tt_write(term, "\033[?1049l", 8); + assert(tt_cell(term, 0, 0)->text == 'n'); + term_free(term); + + /* A reflow that cannot grow its row buffers fails before it changes the + * history, so the old geometry and text survive the failed resize. The + * first resize sizes the buffers, so only the reflow's growth fails. */ + term = tt_new(20, 40); + assert(term); + for (int i = 0; i < 40; i++) { + char line[32]; + int n = snprintf(line, sizeof line, "row %02d 0123456789\r\n", i); + tt_write(term, line, (size_t)n); + } + resized = tt_resize(term, 21, 40); + assert(resized); + int history = tt_history(term); + char *before = tt_history_text(term, 1000); + assert(before); + + fail_reallocs = true; + resized = tt_resize(term, 10, 40); + fail_reallocs = false; + assert(!resized); + assert(tt_cols(term) == 21); + assert(tt_rows(term) == 40); + assert(tt_history(term) == history); + char *after = tt_history_text(term, 1000); + assert(after && !strcmp(before, after)); + free(before); + free(after); + + resized = tt_resize(term, 10, 40); + assert(resized); + assert(tt_cols(term) == 10); + term_free(term); + + puts("lazy screen allocation failures preserve observable terminal state"); + return 0; +#endif +} diff --git a/tests/resize_property_test.c b/tests/resize_property_test.c new file mode 100644 index 0000000..360e640 --- /dev/null +++ b/tests/resize_property_test.c @@ -0,0 +1,156 @@ +#include "term/term.h" +#include "term_api.h" + +#include <stdint.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> + +static uint64_t state; + +static unsigned random_below(unsigned limit) +{ + state ^= state << 13; + state ^= state >> 7; + state ^= state << 17; + return (unsigned)(state % limit); +} + +/* Select from the oldest retained row to the end of the live screen. + * Selection text joins wrapped rows and trims blank cells on other rows. */ +static char *logical_text(struct term *t) +{ + struct term_vector2i size = term_size(t); + term_scroll_to(t, term_scrollback(t).total); + term_select_start(t, (struct term_vector2i){ .x = 0, .y = 0 }, false, TERM_SELECT_CELL); + term_scroll_to(t, 0); + term_select_extend(t, (struct term_vector2i){ .x = size.x - 1, .y = size.y - 1 }, true); + char *text = tt_selection_text(t); + term_select_clear(t); + if (!text) return NULL; + size_t out = 0; + for (size_t i = 0; text[i]; i++) { + if (text[i] == '\n') { + while (out && text[out - 1] == ' ') out--; + } + text[out++] = text[i]; + } + while (out && (text[out - 1] == '\n' || text[out - 1] == ' ')) out--; + text[out] = 0; + return text; +} + +static uint32_t cell_at(struct term *t, struct term_snapshot *s, int x, int y) +{ + if (!term_snapshot_update(s, t)) return 0; + return term_snapshot_row(s, y)[x].text; +} + +/* A height change that pulls rows back from history moves the saved + * cursor with its row. DECRC then prints next to the saved text. */ +static int saved_cursor_follows_history(struct term_snapshot *s) +{ + static const struct { int cols, rows, grow_cols, grow_rows; } cases[] = { + { 8, 3, 8, 5 }, /* same width: rows come back from history */ + { 8, 3, 10, 5 }, /* new width: reflow */ + }; + for (size_t i = 0; i < sizeof(cases) / sizeof(cases[0]); i++) { + struct term_options opt = { + .size = { .x = cases[i].cols, .y = cases[i].rows }, .scrollback = 10, + }; + struct term *t = term_new(&opt); + if (!t) return 1; + const char *text = "l1\r\nl2\r\nl3\r\nl4\r\nab\0337\033[1;1H"; + term_feed(t, text, strlen(text), 0); + struct term_vector2i size = { .x = cases[i].grow_cols, .y = cases[i].grow_rows }; + if (!term_resize(t, size)) return 1; + term_feed(t, "\0338X", 3, 0); + struct term_cursor c = term_cursor(t); + int y = c.pos.y; + if (cell_at(t, s, 0, y) != 'a' || cell_at(t, s, 1, y) != 'b' || + cell_at(t, s, 2, y) != 'X') { + fprintf(stderr, "case %zu: DECRC after a resize lost the saved row\n", i); + return 1; + } + term_free(t); + } + return 0; +} + +/* Narrowing a full screen with the cursor at the top wraps more rows than + * the ring holds. The cursor row must survive and stay on screen. */ +static int narrow_resize_keeps_cursor_row(struct term_snapshot *s) +{ + struct term_options opt = { .size = { .x = 40, .y = 12 }, .scrollback = 5 }; + struct term *t = term_new(&opt); + if (!t) return 1; + for (int y = 0; y < 12; y++) { + char line[64]; + memset(line, 'a' + y, 40); + line[40] = 0; + term_feed(t, line, 40, 0); + } + term_feed(t, "\033[1;1H", 6, 0); + if (!term_resize(t, (struct term_vector2i){ .x = 1, .y = 12 })) return 1; + struct term_cursor c = term_cursor(t); + if (c.pos.x != 0 || c.pos.y != 0 || cell_at(t, s, 0, 0) != 'a') { + fprintf(stderr, "narrow resize: cursor %d,%d, cell %x\n", c.pos.x, c.pos.y, + cell_at(t, s, 0, 0)); + return 1; + } + term_feed(t, "Z", 1, 0); + if (cell_at(t, s, 0, 0) != 'Z') return 1; + term_free(t); + return 0; +} + +int main(int argc, char **argv) +{ + struct term_snapshot *snap = term_snapshot_new(); + if (!snap || saved_cursor_follows_history(snap) || narrow_resize_keeps_cursor_row(snap)) + return 1; + term_snapshot_free(snap); + + int seeds = argc > 1 ? atoi(argv[1]) : 100; + static const char *word[] = { + "alpha", " x", "中", "日", "e\xcc\x81", "🦊", "👍", " beta", "123", "a", + }; + for (int seed = 1; seed <= seeds; seed++) { + state = 0x9e3779b97f4a7c15ull * (uint64_t)seed; + struct term_options opt = { + .size = { .x = 40, .y = 12 }, + .scrollback = 4096, + }; + struct term *t = term_new(&opt); + if (!t) return 1; + for (int line = 0; line < 16; line++) { + int words = 1 + (int)random_below(4); + for (int i = 0; i < words; i++) { + const char *s = word[random_below(sizeof(word) / sizeof(word[0]))]; + term_feed(t, s, strlen(s), 0); + } + if (line != 15) term_feed(t, "\r\n", 2, 0); + } + char *before = logical_text(t); + if (!before) return 1; + for (int step = 0; step < 20; step++) { + struct term_vector2i size = { + .x = 1 + (int)random_below(200), + .y = 1 + (int)random_below(60), + }; + if (!term_resize(t, size)) return 1; + char *after = logical_text(t); + if (!after) return 1; + if (strcmp(before, after)) { + fprintf(stderr, "seed %d step %d resize %dx%d: logical text changed\n" + "before: <%s>\nafter: <%s>\n", + seed, step, size.x, size.y, before, after); + return 1; + } + free(after); + } + free(before); + term_free(t); + } + return 0; +} diff --git a/tests/spec_test.c b/tests/spec_test.c new file mode 100644 index 0000000..ec9d45d --- /dev/null +++ b/tests/spec_test.c @@ -0,0 +1,395 @@ +/* + * Screen specs: one case per control, escape sequence, and CSI final that + * the write core handles. Expected results follow xterm. A case that shows + * a deliberate difference from xterm says so. + * + * Rows read as text: '#' is U+FFFD, '?' is any other code point outside + * ASCII or a cluster, '_' is the right half of a wide glyph. + */ +#include "term/term.h" + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> + +struct spec { + const char *name, *input; + int cols, rows; + const char *line[4]; + int x, y; + /* Optional checks. attr_x < 0 skips the cell check; history < 0 skips + * the history check; reply NULL skips the reply check. */ + int attr_x, attr_y; + uint16_t attrs; + term_color fg, bg; + int history; + const char *reply; +}; + +#define ROWS3(a, b, c) .line = { a, b, c } +#define AT(x_, y_) .x = x_, .y = y_ +#define NO_EXTRA .attr_x = -1, .history = -1 +#define CASE(name_, input_, a, b, c, x_, y_) \ + { .name = name_, .input = input_, .cols = 8, .rows = 3, ROWS3(a, b, c), AT(x_, y_), NO_EXTRA } +#define RED (TERM_COLOR_SLOT | 1) +#define BLUE (TERM_COLOR_SLOT | 4) + +static const struct spec cases[] = { + /* Printing and C0 controls. */ + CASE("print", "abc", "abc", "", "", 3, 0), + CASE("delayed wrap", "abcdefghZ", "abcdefgh", "Z", "", 1, 1), + CASE("carriage return", "abc\rZ", "Zbc", "", "", 1, 0), + CASE("line feed keeps column", "ab\nZ", "ab", " Z", "", 3, 1), + CASE("vertical tab is a line feed", "ab\x0bZ", "ab", " Z", "", 3, 1), + CASE("form feed is a line feed", "ab\x0cZ", "ab", " Z", "", 3, 1), + CASE("backspace", "abc\bZ", "abZ", "", "", 3, 0), + CASE("backspace at column 0", "\bZ", "Z", "", "", 1, 0), + /* The cursor sits on the last column; BS moves left from there. */ + CASE("backspace clears pending wrap", "abcdefgh\bZ", "abcdefZh", "", "", 7, 0), + CASE("tab", "a\tZ", "a Z", "", "", 7, 0), + CASE("tab stops at the last column", "\t\tZ", " Z", "", "", 7, 0), + CASE("tab clears pending wrap", "abcdefgh\tZ", "abcdefgZ", "", "", 7, 0), + { .name = "tab does not erase", .input = "abcdefghijk\r\tZ", .cols = 12, .rows = 2, + .line = { "abcdefghZjk" }, AT(9, 0), NO_EXTRA }, + CASE("tab clear all", "\033[3g\tZ", " Z", "", "", 7, 0), + { .name = "tab clear at cursor", .input = "\033[9G\033[0g\r\tZ", .cols = 12, .rows = 2, + .line = { " Z" }, AT(11, 0), NO_EXTRA }, + { .name = "tab set", .input = "\033[4G\033H\r\tZ", .cols = 12, .rows = 2, + .line = { " Z" }, AT(4, 0), NO_EXTRA }, + CASE("bell", "a\aZ", "aZ", "", "", 2, 0), + CASE("delete is ignored", "ab\x7f" "c", "abc", "", "", 3, 0), + CASE("NUL, CAN, SUB are ignored", "a\x18" "b\x1a" "c", "abc", "", "", 3, 0), + CASE("scroll", "a\r\nb\r\nc\r\nd", "b", "c", "d", 1, 2), + CASE("shift out to DEC graphics", "\033)0a\x0eq\x0fq", "a?q", "", "", 3, 0), + CASE("DEC graphics in G0", "\033(0x\033(Bx", "?x", "", "", 2, 0), + + /* UTF-8. Each maximal invalid subpart prints U+FFFD (Unicode 3.9). */ + CASE("UTF-8 truncated before ASCII", "a\xe3\x80z", "a#z", "", "", 3, 0), + CASE("UTF-8 lone continuation", "a\x80z", "a#z", "", "", 3, 0), + CASE("UTF-8 surrogate", "\xed\xa0\x80z", "###z", "", "", 4, 0), + CASE("UTF-8 overlong", "\xc0\xafz", "##z", "", "", 3, 0), + CASE("UTF-8 bad second byte", "\xf0\x80z", "##z", "", "", 3, 0), + CASE("UTF-8 ended by a control", "\xc3\rZ", "Z", "", "", 1, 0), + CASE("UTF-8 ended by ESC", "\xc3\033[CZ", "# Z", "", "", 3, 0), + CASE("UTF-8 ended by DEL", "\xc3\x7fZ", "#Z", "", "", 2, 0), + CASE("UTF-8 C1 NEL", "ab\xc2\x85Z", "ab", "Z", "", 1, 1), + CASE("UTF-8 C1 CSI", "\xc2\x9b" "3CZ", " Z", "", "", 4, 0), + CASE("wide glyph", "a中b", "a?_b", "", "", 4, 0), + CASE("wide glyph wraps with a pad", "abcdefg中", "abcdefg", "?_", "", 2, 1), + /* A mark joins the glyph left of the cursor, never the row above. */ + CASE("combining mark at column 0 has no base", "abcdefgh\r\n\xcc\x81Z", + "abcdefgh", "Z", "", 1, 1), + CASE("combining mark at right margin", "\033[?69h\033[2;4s\033[1;2Habc\xcc\x81Z", + " ab?", " Z", "", 2, 1), + CASE("combining mark at left margin has no base", "\033[?69h\033[3;6s\033[1;1Habcd\033[1;3H\xcc\x81Z", + "abZd", "", "", 3, 0), + { .name = "wide glyph on one column", .input = "中", .cols = 1, .rows = 1, + .line = { "?" }, AT(0, 0), .attr_x = -1, .history = 0 }, + + /* CSI cursor moves. */ + CASE("CUP", "abc\033[2;3HZ", "abc", " Z", "", 3, 1), + CASE("CUP clamps", "\033[9;9HZ", "", "", " Z", 7, 2), + CASE("CUU", "\033[3;4H\033[2AZ", " Z", "", "", 4, 0), + CASE("CUD", "\033[1;4H\033[2BZ", "", "", " Z", 4, 2), + CASE("CUF", "\033[3CZ", " Z", "", "", 4, 0), + CASE("CUB", "abc\033[2DZ", "aZc", "", "", 2, 0), + CASE("CNL", "\033[2EZ", "", "", "Z", 1, 2), + CASE("CPL", "\033[3;4H\033[2FZ", "Z", "", "", 1, 0), + CASE("CHA", "abc\033[2GZ", "aZc", "", "", 2, 0), + CASE("HPA", "abc\033[2`Z", "aZc", "", "", 2, 0), + CASE("HPR", "a\033[2aZ", "a Z", "", "", 4, 0), + CASE("VPA", "a\033[3dZ", "a", "", " Z", 2, 2), + CASE("VPR", "a\033[2eZ", "a", "", " Z", 2, 2), + /* Vertical moves stop at a margin when they start inside the region. */ + CASE("CUU stops at the top margin", "\033[2;3r\033[3;1H\033[5AZ", "", "Z", "", 1, 1), + CASE("CUD stops at the bottom margin", "\033[1;2r\033[5BZ", "", "Z", "", 1, 1), + { .name = "CUU from below the region", .input = "\033[2;3r\033[4;1H\033[9AZ", + .cols = 8, .rows = 4, .line = { "", "Z", "", "" }, AT(1, 1), NO_EXTRA }, + { .name = "CUD from above the region", .input = "\033[2;3r\033[1;1H\033[9BZ", + .cols = 8, .rows = 4, .line = { "", "", "Z", "" }, AT(1, 2), NO_EXTRA }, + + /* Erase, insert, delete. */ + CASE("EL 0", "abcdef\033[1;3H\033[K", "ab", "", "", 2, 0), + CASE("EL 1", "abcdef\033[1;3H\033[1K", " def", "", "", 2, 0), + CASE("EL 2", "abcdef\033[2K", "", "", "", 6, 0), + CASE("ED 0", "abcd\r\nefgh\033[1;3H\033[J", "ab", "", "", 2, 0), + CASE("ED 1", "abcd\r\nefgh\033[2;3H\033[1J", "", " h", "", 2, 1), + CASE("ECH", "abcdef\033[1;3H\033[2X", "ab ef", "", "", 2, 0), + CASE("ECH erases a whole wide glyph", "a中b\033[1;3H\033[X", "a b", "", "", 2, 0), + CASE("DCH", "abcdef\033[1;3H\033[2P", "abef", "", "", 2, 0), + CASE("ICH", "abcdef\033[1;3H\033[2@", "ab cdef", "", "", 2, 0), + CASE("REP", "a\033[3b", "aaaa", "", "", 4, 0), + CASE("insert mode", "abcd\r\033[4hZ", "Zabcd", "", "", 1, 0), + CASE("insert mode at the row end", "abc\033[4hZ", "abcZ", "", "", 4, 0), + /* IL and DL also move the cursor to the left margin. */ + CASE("IL", "a\r\nb\r\nc\033[2;3H\033[L", "a", "", "b", 0, 1), + CASE("DL", "a\r\nb\r\nc\033[2;3H\033[M", "a", "c", "", 0, 1), + CASE("SU", "a\r\nb\r\nc\033[S", "b", "c", "", 1, 2), + CASE("SD", "a\r\nb\r\nc\033[T", "", "a", "b", 1, 2), + CASE("CHT", "\033[1IZ", " Z", "", "", 7, 0), + CASE("CBT", "\033[2;3H\033[1ZZ", "", "Z", "", 1, 1), + /* ED 2 moves rows up to the last one with a glyph to history. A row + * that DCH emptied does not count. This keeps rows in history where + * xterm does not. */ + { .name = "ED 2 keeps text in history", .input = "x\r\nabc\r\033[3P\033[2J", + .cols = 8, .rows = 3, ROWS3("", "", ""), AT(0, 1), .attr_x = -1, .history = 1 }, + { .name = "ED 2 after ECH reads cells", .input = "x\r\nabc\r\033[3X\033[2J", + .cols = 8, .rows = 3, ROWS3("", "", ""), AT(0, 1), .attr_x = -1, .history = 1 }, + { .name = "ED 3", .input = "a\r\nb\r\nc\r\nd\033[3J", .cols = 8, .rows = 3, + ROWS3("b", "c", "d"), AT(1, 2), .attr_x = -1, .history = 0 }, + + /* ESC sequences. */ + CASE("IND", "a\033D Z", "a", " Z", "", 3, 1), + CASE("NEL", "abc\033EZ", "abc", "Z", "", 1, 1), + CASE("RI", "\033[2;1HZ\033M", "", "Z", "", 1, 0), + CASE("RI scrolls at the top", "a\033MZ", " Z", "a", "", 2, 0), + CASE("save restore", "a\0337bc\0338Z", "aZc", "", "", 2, 0), + CASE("restore without save goes home", "abc\0338Z", "Zbc", "", "", 1, 0), + CASE("restore keeps the pending wrap", "abcdefgh\0337\r\0338Z", "abcdefgh", "Z", "", 1, 1), + CASE("restore restores origin mode", "\033[2;3r\033[?6h\0337\033[?6l\0338\033[HZ", + "", "Z", "", 1, 1), + CASE("restore restores the character set", "\033(0\0337\033(B\0338x", "?", "", "", 1, 0), + CASE("reset", "abc\033cZ", "Z", "", "", 1, 0), + CASE("screen alignment", "\033#8", "EEEEEEEE", "EEEEEEEE", "EEEEEEEE", 0, 0), + + /* Modes. */ + CASE("autowrap off", "\033[?7labcdefghi", "abcdefgi", "", "", 7, 0), + CASE("alternate", "abc\033[?1049hZ", "Z", "", "", 1, 0), + CASE("alternate restores", "abc\033[?1049hZ\033[?1049lQ", "abcQ", "", "", 4, 0), + CASE("origin", "\033[2;3r\033[?6h\033[HZ", "", "Z", "", 1, 1), + CASE("margin", "\033[2;3r\033[3;1Ha\033D", "", "a", "", 1, 2), + CASE("line feed mode", "ab\033[20h\nZ", "ab", "Z", "", 1, 1), + + /* Parsing. C0 controls run inside a CSI (ECMA-48 5.5, xterm). */ + CASE("C0 inside CSI", "\033[2\n;3HZ", "", " Z", "", 3, 1), + CASE("CAN cancels CSI", "\033[31\x18Z", "Z", "", "", 1, 0), + CASE("high byte cancels CSI", "\033[3\xffZ", "#Z", "", "", 2, 0), + CASE("DEL inside CSI is ignored", "\033[\1772CZ", " Z", "", "", 3, 0), + CASE("OSC is swallowed", "a\033]0;title\007b", "ab", "", "", 2, 0), + CASE("DCS is swallowed", "a\033Pxyz\033\\b", "ab", "", "", 2, 0), + CASE("APC is swallowed", "a\033_x\033\\b", "ab", "", "", 2, 0), + + /* Attributes. */ + { + .name = "SGR", .input = "\033[1;3;4;9;53;38;5;123;48;2;1;2;3mX", + .cols = 8, .rows = 3, ROWS3("X", "", ""), AT(1, 0), + .attr_x = 0, .attr_y = 0, + .attrs = TERM_ATTR_BOLD | TERM_ATTR_ITALIC | TERM_ATTR_STRIKE | + TERM_ATTR_OVERLINE | (TERM_UNDERLINE_SINGLE << TERM_ATTR_UNDERLINE_SHIFT), + .fg = TERM_COLOR_SLOT | 123, .bg = 0xff010203u, .history = -1, + }, + { + .name = "SGR colon RGB with a color space", .input = "\033[38:2::1:2:3mX", + .cols = 8, .rows = 3, ROWS3("X", "", ""), AT(1, 0), + .attr_x = 0, .attr_y = 0, .fg = 0xff010203u, .bg = TERM_COLOR_BG, .history = -1, + }, + { + /* 48:2:: takes the next three fields as RGB; the trailing 0 resets. */ + .name = "SGR colon form is not two semicolon colors", + .input = "\033[48:2::::38;2;;;mX", + .cols = 8, .rows = 3, ROWS3("X", "", ""), AT(1, 0), + .attr_x = 0, .attr_y = 0, .fg = TERM_COLOR_FG, .bg = TERM_COLOR_BG, .history = -1, + }, + { + .name = "restore restores the pen", .input = "\033[31m\0337\033[m\0338X", + .cols = 8, .rows = 3, ROWS3("X", "", ""), AT(1, 0), + .attr_x = 0, .attr_y = 0, .fg = RED, .bg = TERM_COLOR_BG, .history = -1, + }, + { + .name = "SCORC restores the pen", .input = "\033[1m\033[s\033[m\033[uX", + .cols = 8, .rows = 3, ROWS3("X", "", ""), AT(1, 0), + .attr_x = 0, .attr_y = 0, .attrs = TERM_ATTR_BOLD, + .fg = TERM_COLOR_FG, .bg = TERM_COLOR_BG, .history = -1, + }, + { + .name = "restore without save resets the pen", .input = "\033[31m\0338X", + .cols = 8, .rows = 3, ROWS3("X", "", ""), AT(1, 0), + .attr_x = 0, .attr_y = 0, .fg = TERM_COLOR_FG, .bg = TERM_COLOR_BG, .history = -1, + }, + { + /* A rectangle edge on the right half of a wide glyph changes both + * halves. */ + .name = "DECCARA on half a wide glyph", .input = "a中b\033[1;3;1;3;1$r", + .cols = 8, .rows = 3, ROWS3("a?_b", "", ""), AT(4, 0), + .attr_x = 1, .attr_y = 0, .attrs = TERM_ATTR_BOLD, + .fg = TERM_COLOR_FG, .bg = TERM_COLOR_BG, .history = -1, + }, + { + .name = "erase uses the pen background", .input = "ab\033[41m\033[1;1H\033[X", + .cols = 8, .rows = 3, ROWS3(" b", "", ""), AT(0, 0), + .attr_x = 0, .attr_y = 0, .fg = TERM_COLOR_FG, .bg = TERM_COLOR_SLOT | 1, .history = -1, + }, + { + .name = "full-screen scroll BCE", .input = "abc\033[44m\r\n\r\n\r\n", + .cols = 8, .rows = 3, ROWS3("", "", ""), AT(0, 2), + .attr_x = 0, .attr_y = 2, + .fg = TERM_COLOR_FG, .bg = BLUE, .history = -1, + }, + + /* Replies. */ + { .name = "DSR 5", .input = "\033[5n", .cols = 8, .rows = 3, ROWS3("", "", ""), + AT(0, 0), NO_EXTRA, .reply = "\033[0n" }, + { .name = "DSR 6", .input = "ab\033[6n", .cols = 8, .rows = 3, ROWS3("ab", "", ""), + AT(2, 0), NO_EXTRA, .reply = "\033[1;3R" }, + /* The cursor stays on the last column while a wrap is pending. */ + { .name = "DSR 6 with a pending wrap", .input = "abcdefgh\033[6n", .cols = 8, .rows = 3, + ROWS3("abcdefgh", "", ""), AT(7, 0), NO_EXTRA, .reply = "\033[1;8R" }, + { .name = "DSR 6 in origin mode", .input = "\033[2;3r\033[?6h\033[6n", .cols = 8, .rows = 3, + ROWS3("", "", ""), AT(0, 1), NO_EXTRA, .reply = "\033[1;1R" }, + { .name = "DA1", .input = "\033[c\033[0c", .cols = 8, .rows = 3, ROWS3("", "", ""), + AT(0, 0), NO_EXTRA, .reply = "\033[?62;22c\033[?62;22c" }, + { .name = "DA1 with a nonzero parameter", .input = "\033[1c", .cols = 8, .rows = 3, + ROWS3("", "", ""), AT(0, 0), NO_EXTRA, .reply = "" }, + { .name = "DA2", .input = "\033[>c", .cols = 8, .rows = 3, ROWS3("", "", ""), + AT(0, 0), NO_EXTRA, .reply = "\033[>0;10;0c" }, + { .name = "DECRQSS SGR", .input = "\033[1;31m\033P$qm\033\\", .cols = 8, .rows = 3, + ROWS3("", "", ""), AT(0, 0), NO_EXTRA, .reply = "\033P1$r0;1;31m\033\\" }, + /* SGR 58 has no short form for colors 0 to 15. */ + { .name = "DECRQSS underline color", .input = "\033[58;5;3m\033P$qm\033\\", + .cols = 8, .rows = 3, ROWS3("", "", ""), AT(0, 0), NO_EXTRA, + .reply = "\033P1$r0;58:5:3m\033\\" }, + { .name = "DECRQSS margins", .input = "\033[2;3r\033P$qr\033\\", .cols = 8, .rows = 3, + ROWS3("", "", ""), AT(0, 0), NO_EXTRA, .reply = "\033P1$r2;3r\033\\" }, + { .name = "DECRQM", .input = "\033[4h\033[4$p\033[?7$p", .cols = 8, .rows = 3, + ROWS3("", "", ""), AT(0, 0), NO_EXTRA, .reply = "\033[4;1$y\033[?7;1$y" }, +}; + +struct replies { + char s[512]; + size_t len; +}; + +static void on_reply(void *ud, const char *bytes, size_t len) +{ + struct replies *r = ud; + if (r->len + len < sizeof(r->s)) { + memcpy(r->s + r->len, bytes, len); + r->len += len; + } +} + +static char cell_char(const struct term_snapshot *s, const struct term_cell *c) +{ + if (c->width == 0) return '_'; + uint32_t cps[4] = {0}; + int n = term_snapshot_cluster(s, c->text, cps, 4); + if (n != 1) return '?'; + if (cps[0] == 0xFFFD) return '#'; + return cps[0] >= 0x20 && cps[0] < 0x7f ? (char)cps[0] : '?'; +} + +static int run(const struct spec *c, struct term_snapshot *s, bool bytes) +{ + struct replies got = {0}; + struct term_options opt = { + .size = { .x = c->cols, .y = c->rows }, .scrollback = 100, + .callbacks = { .ud = &got, .reply = on_reply }, + }; + struct term *t = term_new(&opt); + if (!t) return 1; + size_t len = strlen(c->input); + if (bytes) + for (size_t i = 0; i < len; i++) term_feed(t, c->input + i, 1, 0); + else + term_feed(t, c->input, len, 0); + const char *mode = bytes ? "byte by byte" : "whole"; + int bad = 0; + struct term_cursor cursor = term_cursor(t); + if (cursor.pos.x != c->x || cursor.pos.y != c->y) { + fprintf(stderr, "%s (%s): cursor %d,%d; expected %d,%d\n", c->name, mode, + cursor.pos.x, cursor.pos.y, c->x, c->y); + bad = 1; + } + if (!term_snapshot_update(s, t)) return 1; + for (int y = 0; y < c->rows && y < 4; y++) { + const struct term_cell *row = term_snapshot_row(s, y); + char text[64]; + int n = 0; + for (int x = 0; x < c->cols && n < 63; x++) text[n++] = cell_char(s, &row[x]); + while (n > 0 && text[n - 1] == ' ') n--; + text[n] = 0; + const char *want = c->line[y] ? c->line[y] : ""; + if (strcmp(text, want)) { + fprintf(stderr, "%s (%s): row %d \"%s\"; expected \"%s\"\n", c->name, mode, + y, text, want); + bad = 1; + } + } + if (c->attr_x >= 0) { + const struct term_cell *cell = &term_snapshot_row(s, c->attr_y)[c->attr_x]; + if (cell->attrs != c->attrs || cell->fg != c->fg || cell->bg != c->bg) { + fprintf(stderr, "%s (%s): attributes %x fg %x bg %x; expected %x %x %x\n", + c->name, mode, cell->attrs, cell->fg, cell->bg, c->attrs, c->fg, c->bg); + bad = 1; + } + } + if (c->history >= 0 && term_scrollback(t).total != c->history) { + fprintf(stderr, "%s (%s): history %d; expected %d\n", c->name, mode, + term_scrollback(t).total, c->history); + bad = 1; + } + if (c->reply && (got.len != strlen(c->reply) || memcmp(got.s, c->reply, got.len))) { + fprintf(stderr, "%s (%s): reply differs\n", c->name, mode); + bad = 1; + } + term_free(t); + return bad; +} + +int main(void) +{ + struct term_snapshot *s = term_snapshot_new(); + if (!s) return 1; + int failed = 0; + for (size_t i = 0; i < sizeof(cases) / sizeof(cases[0]); i++) + failed += run(&cases[i], s, false) | run(&cases[i], s, true); + + /* ONLCR is a feed option. A plain line feed keeps the column. */ + struct term_options opt = { .size = { .x = 8, .y = 2 } }; + struct term *t = term_new(&opt); + if (!t) return 1; + term_feed(t, "ab\nZ", 4, TERM_FEED_ONLCR); + if (term_cursor(t).pos.x != 1 || term_cursor(t).pos.y != 1) { + fprintf(stderr, "ONLCR: line feed did not return to column zero\n"); + failed++; + } + term_free(t); + + /* A long stream can skip old rows. A bare CR must keep the last rows. */ + opt.size = (struct term_vector2i){ .x = 8, .y = 3 }; + opt.scrollback = 5; + t = term_new(&opt); + if (!t) return 1; + char *stream = malloc(13000u * 9u + 8u); + if (!stream) return 1; + for (int i = 0; i < 13000; i++) memcpy(stream + (size_t)i * 9, "1234567\r\n", 9); + memcpy(stream + 13000u * 9u, "broken\rY", 8); + term_feed(t, stream, 13000u * 9u + 8u, 0); + free(stream); + if (term_scrollback(t).total != 5) { + fprintf(stderr, "long stream: history count differs\n"); + failed++; + } + for (int offset = 5;; offset = offset > 3 ? offset - 3 : 0) { + term_scroll_to(t, offset); + if (!term_snapshot_update(s, t)) return 1; + for (int y = 0; y < 3; y++) { + const char *want = offset == 0 && y == 2 ? "Yroken" : "1234567"; + const struct term_cell *row = term_snapshot_row(s, y); + for (int x = 0; x < 8; x++) { + uint32_t cp = x < (int)strlen(want) ? (unsigned char)want[x] : ' '; + if (row[x].text != cp) { + fprintf(stderr, "long stream: offset %d cell %d,%d = %x, expected %x\n", + offset, x, y, row[x].text, cp); + failed++; + break; + } + } + } + if (!offset) break; + } + term_free(t); + term_snapshot_free(s); + if (failed) fprintf(stderr, "%d spec failures\n", failed); + return failed != 0; +} diff --git a/tests/symbols_test.sh b/tests/symbols_test.sh new file mode 100644 index 0000000..4c7aee1 --- /dev/null +++ b/tests/symbols_test.sh @@ -0,0 +1,63 @@ +#!/usr/bin/env bash +# Every global symbol of the term library starts with term_. A host links +# the library next to its own code, so a bare name such as palette_init can +# collide with a host symbol. Every function that term.h declares is also +# defined, so the library cannot leave one out. A shared build exports +# exactly the functions of term.h. +set -euo pipefail +lib=$1 +src=${2:-$(dirname "$0")/../include/term} + +syms=$(nm -g --defined-only -P "$lib") +# ASan adds an __odr_asan.<name> symbol for each global variable. +defined=$(awk 'NF >= 2 && $2 ~ /^[A-Z]$/ && $2 != "U" { print $1 }' <<<"$syms" | + grep -v '^_*odr_asan\.' | sed 's/^_//' | sort -u) + +status=0 +bad=$(grep -v '^term_' <<<"$defined" || true) +if [[ -n $bad ]]; then + echo "$lib: global symbols without the term_ prefix:" >&2 + echo "$bad" >&2 + status=1 +fi + +declared=$("${CC:-cc}" -std=c11 -E -P -I"$src" "$src/term.h" | python3 -c ' +import re, sys +text = re.sub(r"\s+", " ", sys.stdin.read()) +depth, flat = 0, [] +for ch in text: + depth += ch == "{" + if depth == 0: + flat.append(ch) + depth -= ch == "}" + if ch == "}" and depth == 0: + flat.append(";") +for stmt in "".join(flat).split(";"): + if "static" in stmt or "typedef" in stmt: + continue + m = re.search(r"\b(term_\w+)\s*\(", stmt) + if m: + print(m.group(1)) +' | sort -u) +missing=$(comm -23 <(echo "$declared") <(echo "$defined")) +if [[ -n $missing ]]; then + echo "$lib: functions declared in term.h but not defined:" >&2 + echo "$missing" >&2 + status=1 +fi + +# The shared library exports term.h and nothing else. +shared=${3:-} +if [[ -n $shared ]]; then + exported=$(nm -D --defined-only -P "$shared" | awk '$2 ~ /^[TDBRVW]$/ { print $1 }' | + sed 's/^_//' | sort -u) + extra=$(comm -13 <(echo "$declared") <(echo "$exported")) + absent=$(comm -23 <(echo "$declared") <(echo "$exported")) + if [[ -n $extra || -n $absent ]]; then + echo "$shared: exports differ from term.h" >&2 + [[ -n $extra ]] && echo "not in term.h: $extra" >&2 + [[ -n $absent ]] && echo "not exported: $absent" >&2 + status=1 + fi +fi +exit $status diff --git a/tests/term_test.c b/tests/term_test.c new file mode 100644 index 0000000..02c050e --- /dev/null +++ b/tests/term_test.c @@ -0,0 +1,2369 @@ +#include "term_api.h" +#include "term/unicode/unicode.h" +#include "term/unicode/utf8.h" +#include <assert.h> +#include <limits.h> +#include <pthread.h> +#include <stdatomic.h> +#include <stdbool.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> + +static char response[64]; +static size_t response_len; +static void respond(void *ud, const char *data, size_t len) +{ + (void)ud; + assert(len < sizeof(response)); + memcpy(response, data, len); + response[len] = 0; + response_len = len; +} +static char title[128]; +static void capture_title(void *ud, const char *s) +{ + (void)ud; + snprintf(title, sizeof(title), "%s", s); +} +static const struct term_callbacks reply_cb = { .reply = respond }; +static const struct term_callbacks title_cb = { .reply = respond, .title = capture_title }; + +static void write_seq(struct term *t, const char *s) +{ + + for (; *s; s++) tt_write(t, s, 1); +} +static void feed_seq(struct term *t, const char *s, unsigned flags) +{ + for (; *s; s++) term_feed(t, s, 1, flags); +} +static const char *row_text(struct term *t, int r, char *buf) +{ + const struct term_cell *cells = term_snapshot_row(tt_snapshot_for(t), r); + int n = 0; + for (int c = 0; c < tt_cols(t); c++) { + uint32_t cp = tt_cell_cp(t, &cells[c]); + /* '?' stands for any code point outside ASCII. */ + buf[n++] = !cp ? ' ' : cp < 0x80 ? (char)cp : '?'; + } + while (n > 0 && buf[n - 1] == ' ') n--; + buf[n] = 0; + return buf; +} +static bool row_eq(struct term *t, int r, const char *want) +{ + char *got = malloc((size_t)tt_cols(t) + 1); + assert(got); + bool equal = !strcmp(row_text(t, r, got), want); + free(got); + return equal; +} +static bool row_is(struct term *t, int r, const char *want) +{ + char *got = malloc((size_t)tt_cols(t) + 1); + assert(got); + bool equal = !strcmp(row_text(t, r, got), want); + if (!equal) + fprintf(stderr, "row %d: got \"%s\" want \"%s\"\n", r, got, want); + free(got); + return equal; +} + +static void write_both(struct term *whole, struct term *bytes, const char *s) +{ + tt_write(whole, s, strlen(s)); + write_seq(bytes, s); +} + +/* The view version of s after it copies t. A renderer redraws when the + * version changes. */ +static uint64_t view_version(struct term_snapshot *s, struct term *t) +{ + term_snapshot_update(s, t); + return term_snapshot_view(s)->version; +} + +/* + * Regression: diff renderers (Claude Code, Ink) skip unchanged spaces with + * CUF. Cells the cursor skips must read as blanks, even on a row whose cells + * still hold old glyphs: a row recycled by scrolling, or a row whose tail + * an erase dropped without filling. Stale glyphs showed up in place of + * every skipped space. + */ +static void test_skipped_cells_read_blank(void) +{ + static const char *const lines[] = { + "ab\033[1Ccd\033[3Cef", + "a…b\033[1Cc", + "\033[1m\033[38;2;1;2;3mab\033[0m\033[2Ccd", + }; + /* row_text() reads U+2026 as '?'. */ + static const char *const want[] = { "ab cd ef", "a?b c", "ab cd" }; + for (size_t k = 0; k < sizeof(lines) / sizeof(lines[0]); k++) { + struct term *whole = tt_new(20, 4); + struct term *bytes = tt_new(20, 4); + for (int i = 0; i < 12; i++) + write_both(whole, bytes, "XXXXXXXXXXXXXXXXXXXX\r\n"); + write_both(whole, bytes, lines[k]); + for (struct term *t = whole; t; t = t == whole ? bytes : NULL) { + char *got = malloc((size_t)tt_cols(t) + 1); + assert(got); + row_text(t, 3, got); + if (strcmp(got, want[k])) { + fprintf(stderr, "line %zu (%s): got \"%s\" want \"%s\"\n", k, + t == whole ? "whole" : "bytes", got, want[k]); + assert(0); + } + free(got); + } + term_free(whole); + term_free(bytes); + } + + struct term *whole = tt_new(20, 4); + struct term *bytes = tt_new(20, 4); + write_both(whole, bytes, "abcdefgh\r\033[2C\033[K\033[2CZ"); + assert(row_is(whole, 0, "ab Z")); + assert(row_is(bytes, 0, "ab Z")); + term_free(whole); + term_free(bytes); +} + +/* Compare the first rows of both viewports cell by cell. Each terminal gets + * its own snapshot. A cluster has a different handle in each snapshot, so + * the test compares its code points. */ +static void assert_same_rows(struct term *a, struct term *b, int rows) +{ + struct term_snapshot *sa = term_snapshot_new(); + struct term_snapshot *sb = term_snapshot_new(); + bool copied = sa && sb && term_snapshot_update(sa, a) && term_snapshot_update(sb, b); + assert(copied); + int cols = term_snapshot_view(sa)->size.x; + assert(cols == term_snapshot_view(sb)->size.x); + for (int y = 0; y < rows; y++) { + const struct term_cell *ra = term_snapshot_row(sa, y); + const struct term_cell *rb = term_snapshot_row(sb, y); + assert(ra && rb); + for (int x = 0; x < cols; x++) { + const struct term_cell *ac = &ra[x], *bc = &rb[x]; + uint32_t acp[32], bcp[32]; + int an = term_snapshot_cluster(sa, ac->text, acp, 32); + int bn = term_snapshot_cluster(sb, bc->text, bcp, 32); + bool same = an == bn && an <= 32 && + !memcmp(acp, bcp, (size_t)an * sizeof(*acp)) && + ac->fg == bc->fg && ac->bg == bc->bg && + ac->attrs == bc->attrs && ac->width == bc->width; + if (!same) { + fprintf(stderr, "x=%d y=%d: a(cp=%x fg=%x bg=%x attrs=%x width=%d) " + "b(cp=%x fg=%x bg=%x attrs=%x width=%d)\n", x, y, + an ? acp[0] : 0, ac->fg, ac->bg, ac->attrs, ac->width, + bn ? bcp[0] : 0, bc->fg, bc->bg, bc->attrs, bc->width); + assert(0); + } + } + } + term_snapshot_free(sa); + term_snapshot_free(sb); +} + +static void assert_same_screen(struct term *a, struct term *b) +{ + assert(tt_cursor_x(a) == tt_cursor_x(b)); + assert(tt_cursor_y(a) == tt_cursor_y(b)); + assert(tt_history(a) == tt_history(b)); + assert(term_modes(a).synchronized == term_modes(b).synchronized); + assert(tt_rows(a) == tt_rows(b)); + assert_same_rows(a, b, tt_rows(a)); +} + +static void test_read_boundaries(void) +{ + static const char input[] = + "ASCII 1234567890\r\nT\xc3\xbc" "rk\xe2\x94\x80" + "\033[31mred\033[m\033(0lqqk\033(B" + "\033[2;3H!\033[2b\033[1D?\033[K" + "\033[?1049halt\033[?1049lZ" + "\033]11;?\007\033Pignored\033\\" + "\xf0\x9f\x98\x80\xc3XQ©️界\033[2G\033[P"; + struct term *expected = tt_new(16, 4); + tt_write(expected, input, sizeof(input) - 1); + for (size_t split = 0; split < sizeof(input); split++) { + struct term *actual = tt_new(16, 4); + tt_write(actual, input, split); + tt_write(actual, input + split, sizeof(input) - 1 - split); + assert_same_screen(actual, expected); + term_free(actual); + } + + /* The byte that ends an unfinished sequence is not lost. */ + struct term *t = tt_new(16, 4); + write_seq(t, "\xc3XQ"); + assert(tt_cell_cp(t, tt_cell(t, 0, 0)) == 0xFFFD); + assert(tt_cell_cp(t, tt_cell(t, 1, 0)) == 'X'); + assert(tt_cell_cp(t, tt_cell(t, 2, 0)) == 'Q'); + term_free(t); + term_free(expected); +} + +static void test_ascii_controls(void) +{ + static const char *separators[] = {"\n", "\r", "\b", "\t", "\033[31m", "\177", "é界"}; + for (int bottom = 0; bottom < 2; bottom++) { + for (size_t c = 0; c < sizeof(separators) / sizeof(separators[0]); c++) { + char input[256]; + memset(input, 'a', 29); + size_t n = strlen(separators[c]); + memcpy(input + 29, separators[c], n); + memset(input + 29 + n, 'b', 173); + input[202 + n] = '\0'; + struct term *bulk = tt_new(120, 6); + struct term *bytes = tt_new(120, 6); + const char *setup = bottom ? "\033[6;1H\033[1;3;4;38;2;12;34;56m" + : "\033[1;3;4;38;2;12;34;56m"; + write_seq(bulk, setup); + write_seq(bytes, setup); + tt_write(bulk, input, strlen(input)); + write_seq(bytes, input); + assert_same_screen(bulk, bytes); + term_scroll(bulk, 6); + term_scroll(bytes, 6); + assert_same_screen(bulk, bytes); + term_free(bulk); + term_free(bytes); + } + } +} + +static void test_esc_control_strings_and_designators(void) +{ + struct term *t = tt_new(32, 2); + assert(t); + + write_seq(t, "A\033_payload ignored\033\\B"); + write_seq(t, "\033^privacy message\007C"); + write_seq(t, "\033Xsos ignored\033\\D"); + assert(row_is(t, 0, "ABCD")); + + write_seq(t, "\033%Gx\033%@y\033%/z"); + assert(row_is(t, 0, "ABCDxyz")); + write_seq(t, "\033%qx"); + assert(row_is(t, 0, "ABCDxyz%qx")); + + write_seq(t, "\033.Am\033*0n"); + assert(row_is(t, 0, "ABCDxyz%qxmn")); + term_free(t); + + t = tt_new(8, 2); + write_seq(t, "A\033_ignored\033DB"); + assert(row_is(t, 0, "A")); + assert(row_is(t, 1, " B")); + term_free(t); +} + +static void test_random_read_boundaries(void) +{ + static const char embedded[] = + "\033\\\033(B\r\nA\xc3\xbc\xe7\x95\x8c" "e\xcc\x81" + "\033[31mR\033[m\tX\bY\033]0;title\007\033Pignored\033\\" + "\033[?2026hS\033[?2026l\xc3XQ\xe0\x80\x80\xf4\x90\x80\x80"; + static const size_t chunks[] = {1, 2, 7, 31, 127}; + for (unsigned seed = 1; seed <= 4; seed++) { + char input[1024]; + unsigned random = seed; + for (size_t i = 0; i < sizeof(input); i++) { + random = random * 1103515245u + 12345u; + input[i] = (char)(random >> 16); + } + memcpy(input + 400, embedded, sizeof(embedded) - 1); + struct term *whole = tt_new(32, 8); + tt_write(whole, input, sizeof(input)); + for (size_t c = 0; c < sizeof(chunks) / sizeof(chunks[0]); c++) { + struct term *a = tt_new(32, 8); + struct term *b = tt_new(32, 8); + for (size_t block = 0; block < sizeof(input); block += 128) { + tt_write(a, input + block, 128); + for (size_t i = block; i < block + 128;) { + size_t n = chunks[c]; + if (n > block + 128 - i) n = block + 128 - i; + tt_write(b, input + i, n); + i += n; + } + assert_same_screen(a, b); + } + assert_same_screen(whole, a); + write_seq(a, "\033\\\033(B\r\nEND"); + write_seq(b, "\033\\\033(B\r\nEND"); + assert_same_screen(a, b); + int history = tt_history(a); + for (int i = 0; i < history; i += 8) { + term_scroll(a, 8); + term_scroll(b, 8); + assert_same_screen(a, b); + } + term_free(a); + term_free(b); + } + term_free(whole); + } +} + +static void test_csi_recovery(void) +{ + static const struct { const char *input, *first, *second; } cases[] = { + /* CR and LF run inside the CSI; B ends it as CUD (ECMA-48 5.5). */ + {"A\033[\r\nB!", "A", ""}, + /* 0xFF cancels the CSI and prints U+FFFD. */ + {"A\033[\xff" "BC", "A?BC", ""}, + {"A\033[\033[2JZ", " Z", ""}, + {"\033[ 31mX", "X", ""}, + {"\033[31$mX", "X", ""}, + {"\033[31;?mX", "X", ""}, + {"\033[31;0;0;0;0;0;0;0;0;0;0;0;0;0;0;0;0mX", "X", ""}, + {"\033[31\030X", "X", ""}, + {"\033[31\032X", "X", ""}, + {"\033[1$ A X", " X", ""}, + {"AB\033[?2JX", "ABX", ""}, + {"AB\r\033[>4hX", "XB", ""}, + }; + for (size_t c = 0; c < sizeof(cases) / sizeof(cases[0]); c++) { + size_t len = strlen(cases[c].input); + for (size_t split = 0; split <= len; split++) { + struct term *t = tt_new(20, 4); + tt_write(t, cases[c].input, split); + tt_write(t, cases[c].input + split, len - split); + assert(row_is(t, 0, cases[c].first)); + assert(row_is(t, 1, cases[c].second)); + for (int x = 0; x < 20; x++) assert(tt_cell(t, x, 0)->fg == TERM_COLOR_FG); + term_free(t); + } + } +} + +static void test_csi_byte_boundaries(void) +{ + static const char *prefixes[] = {"\033[", "\033[31;", "\033[?", "\033[1 ", "\033[1$ "}; + for (size_t p = 0; p < sizeof(prefixes) / sizeof(prefixes[0]); p++) { + for (unsigned byte = 0; byte < 256; byte++) { + char input[64]; + size_t n = strlen(prefixes[p]); + memcpy(input, prefixes[p], n); + input[n++] = (char)byte; + static const char suffix[] = "mX\033[6n\033[?25l\033[?25h\033[0mY"; + memcpy(input + n, suffix, sizeof(suffix) - 1); + n += sizeof(suffix) - 1; + struct term *whole = tt_new_opts(20, 4, &reply_cb); + response_len = 0; + tt_write(whole, input, n); + char expected_response[64]; + size_t expected_len = response_len; + memcpy(expected_response, response, expected_len); + for (size_t split = 0; split <= strlen(prefixes[p]) + 2; split++) { + struct term *parts = tt_new_opts(20, 4, &reply_cb); + response_len = 0; + tt_write(parts, input, split); + tt_write(parts, input + split, n - split); + assert_same_screen(whole, parts); + assert(term_cursor(whole).visible == term_cursor(parts).visible); + assert(response_len == expected_len); + assert(!memcmp(response, expected_response, expected_len)); + term_free(parts); + } + term_free(whole); + } + } +} + +static void test_parameter_limits(void) +{ + struct term *t = tt_new(10, 4); + write_seq(t, "\033[2;4r\033[?6h\033[999999999999999999999;1H"); + assert(tt_cursor_y(t) == 3 && tt_cursor_x(t) == 0); + write_seq(t, "\033[?6l\033[r"); + write_seq(t, "\033[2;2H\033[999999999999999999999B\033[2147483647C!"); + assert(tt_cursor_y(t) == 3 && tt_cursor_x(t) == 9); + assert(tt_cell(t, 9, 3)->text == '!'); + write_seq(t, "\033[999999999999999999999A\033[999999999999999999999D"); + assert(tt_cursor_y(t) == 0 && tt_cursor_x(t) == 0); + write_seq(t, "\033[38;5;999999999999999999999mX"); + assert(tt_cell(t, 0, 0)->fg == TERM_COLOR_FG); + + /* A renderer sees the cursor move to another row. */ + struct term_snapshot *snap = term_snapshot_new(); + uint64_t version = view_version(snap, t); + write_seq(t, "\033[3d"); + assert(tt_cursor_y(t) == 2 && view_version(snap, t) != version); + term_snapshot_free(snap); + + write_seq(t, "\033[4;8H\0337"); + tt_resize(t, 5, 2); + write_seq(t, "\0338Z"); + assert(tt_cursor_y(t) == 1 && tt_cursor_x(t) == 3); + assert(tt_cell(t, 2, 1)->text == 'Z'); + term_free(t); +} + +static void test_unterminated_control_strings_do_not_swallow_text(void) +{ + struct term *t = tt_new(20, 4); + assert(t); + + write_seq(t, "A\033]0;random bytes\tgit\r\nstatus"); + assert(row_is(t, 0, "A git")); + assert(row_is(t, 1, "status")); + + write_seq(t, "\033Prandom bytes\tbranch\r\nnext"); + assert(row_is(t, 1, "status branch")); + assert(row_is(t, 2, "next")); + + term_free(t); + + char overlong[1200]; + for (int intro = 0; intro < 2; intro++) { + int n = snprintf(overlong, sizeof(overlong), intro ? "\033P" : "\033]0;"); + memset(overlong + n, 'T', 1024); + overlong[n + 1024] = '\0'; + + struct term *u = tt_new(20, 4); + write_seq(u, overlong); + write_seq(u, "\r\n[sudo] password:"); + assert(row_is(u, 0, "")); + assert(row_is(u, 1, "[sudo] password:")); + term_free(u); + + struct term *d = tt_new(20, 4); + write_seq(d, overlong); + write_seq(d, "\033\\after"); + assert(row_is(d, 0, "after")); + term_free(d); + } +} + +static void test_c0_controls_after_esc_are_executed(void) +{ + struct term *t = tt_new(20, 4); + assert(t); + + write_seq(t, "A\033\tB"); + assert(row_is(t, 0, "A B")); + + term_free(t); +} + +static void test_resize_reflow(void) +{ + struct term *t = tt_new(8, 4); + write_seq(t, "abcdefghijkl"); + tt_resize(t, 16, 4); + assert(row_is(t, 0, "abcdefghijkl") && row_is(t, 1, "")); + assert(tt_cursor_y(t) == 0 && tt_cursor_x(t) == 12); + write_seq(t, "M"); + assert(row_is(t, 0, "abcdefghijklM")); + tt_resize(t, 5, 4); + assert(row_is(t, 0, "abcde") && row_is(t, 1, "fghij") && row_is(t, 2, "klM")); + assert(tt_cursor_y(t) == 2 && tt_cursor_x(t) == 3); + tt_select(t, 0, 0, 2, 2); + char *text = tt_selection_text(t); + assert(text && !strcmp(text, "abcdefghijklM")); + free(text); + tt_resize(t, 20, 4); + assert(row_is(t, 0, "abcdefghijklM")); + term_free(t); + + t = tt_new(6, 6); + write_seq(t, "\033[31mab cdef\033[m\r\nnext"); + term_color red = tt_cell(t, 0, 0)->fg; + tt_resize(t, 12, 6); + assert(row_is(t, 0, "ab cdef") && row_is(t, 1, "next")); + assert(tt_cell(t, 7, 0)->fg == red); + assert(tt_cell(t, 0, 1)->fg == TERM_COLOR_FG); + term_free(t); + + t = tt_new(4, 2); + write_seq(t, "abcdefghijkl"); + assert(tt_history(t) == 1); + tt_resize(t, 12, 2); + assert(tt_history(t) == 0 && row_is(t, 0, "abcdefghijkl")); + assert(tt_cursor_y(t) == 0 && tt_cursor_x(t) == 11); + write_seq(t, "M"); + assert(row_is(t, 1, "M")); + tt_resize(t, 4, 2); + assert(tt_history(t) == 2); + assert(row_is(t, 0, "ijkl") && row_is(t, 1, "M")); + tt_resize(t, 20, 2); + assert(row_is(t, 0, "abcdefghijklM")); + term_free(t); + + t = tt_new(4, 3); + write_seq(t, "abcd"); + tt_resize(t, 8, 3); + write_seq(t, "E"); + assert(row_is(t, 0, "abcdE")); + tt_resize(t, 5, 3); + write_seq(t, "F"); + assert(row_is(t, 0, "abcdE") && row_is(t, 1, "F")); + term_free(t); + + t = tt_new(4, 4); + write_seq(t, "abcdefghijkl\033[?1049h12345678"); + tt_resize(t, 8, 4); + assert(row_is(t, 0, "1234") && row_is(t, 1, "5678")); + tt_resize(t, 16, 4); + write_seq(t, "\033[?1049lM"); + assert(row_is(t, 0, "abcdefghijklM")); + assert(tt_cursor_y(t) == 0 && tt_cursor_x(t) == 13); + term_free(t); + + t = tt_new(4, 2); + write_seq(t, "abcd"); + tt_resize(t, 4, 3); + write_seq(t, "E"); + assert(row_is(t, 0, "abcd") && row_is(t, 1, "E")); + term_free(t); + + t = tt_new(80, 4); + for (int i = 0; i < 1000; i++) write_seq(t, "abcdefgh"); + tt_resize(t, 1, 4); + assert(tt_history(t) == 1000 * 8 - 4); + assert(tt_cursor_y(t) == 3 && tt_cursor_x(t) == 0); + assert(row_is(t, 0, "e") && row_is(t, 3, "h")); + tt_resize(t, 80, 4); + assert(tt_cursor_x(t) < 80 && tt_cursor_y(t) < 4); + write_seq(t, "Z"); + assert(tt_cell(t, 0, 3)->text == 'Z'); + term_free(t); + +} + +static void test_resize_exposes_blank_rows(void) +{ + struct term *t = tt_new(20, 2); + write_seq(t, "kept"); + for (int pass = 0; pass < 2; pass++) { + tt_resize(t, 20, 6); + assert(row_is(t, 0, "kept")); + for (int y = 2; y < 6; y++) { + for (int x = 0; x < 20; x++) { + const struct term_cell *cell = tt_cell(t, x, y); + assert(cell->text == ' ' && cell->fg == TERM_COLOR_FG); + assert(cell->bg == TERM_COLOR_BG && cell->attrs == 0 && cell->width == 1); + } + } + write_seq(t, "\033[6;1H\tfile/"); + assert(row_is(t, 5, " file/")); + write_seq(t, "\033[1;1H"); + tt_resize(t, 20, 2); + } + term_free(t); +} + +static void test_prompt_resize(void) +{ + struct term *t = tt_new(80, 6); + const char *prompt = "~/Documents/GitHub/clernen_pi main> "; + write_seq(t, "\033]133;A;click_events=1\033\\"); + write_seq(t, prompt); + write_seq(t, "\033]133;B\033\\"); + + const struct { int cols; const char *text; bool newline; } cases[] = { + {40, "~/Documents/GitHub/clernen_pi main> ", false}, + {20, ".b/clernen_pi main> ", true}, + {12, ".n_pi main> ", true}, + {30, "~/GitHub/clernen_pi> ", false}, + {80, "~/Documents/GitHub/clernen_pi main> ", false}, + }; + bool newline = false; + for (size_t i = 0; i < sizeof(cases) / sizeof(cases[0]); i++) { + tt_resize(t, cases[i].cols, 6); + write_seq(t, "\r\r"); + if (newline) write_seq(t, "\033[A"); + write_seq(t, "\033]133;A;click_events=1\033\\"); + write_seq(t, cases[i].text); + write_seq(t, "\033]133;B\033\\"); + if (cases[i].newline) write_seq(t, "\r\n"); + write_seq(t, "\033[J"); + char expected[80]; + snprintf(expected, sizeof(expected), "%s", cases[i].text); + expected[strlen(expected) - 1] = 0; + assert(row_is(t, 0, expected)); + for (int y = 1; y < 6; y++) assert(row_is(t, y, "")); + assert(tt_history(t) == 0); + newline = cases[i].newline; + } + + write_seq(t, "\033]133;C\033\\"); + tt_resize(t, 20, 6); + assert(row_is(t, 0, "~/Documents/GitHub/c")); + assert(row_is(t, 1, "lernen_pi main>")); + term_free(t); + + t = tt_new(20, 6); + write_seq(t, "abcdefghijklmnop\r\n\033]133;A\007prompt> input\033]133;B\007"); + tt_resize(t, 8, 6); + assert(row_is(t, 0, "abcdefgh") && row_is(t, 1, "ijklmnop")); + assert(row_is(t, 2, "prompt>")); + tt_resize(t, 30, 6); + assert(row_is(t, 0, "abcdefghijklmnop") && row_is(t, 1, "prompt>")); + term_free(t); +} + +static void test_cursor_line_motion(void) +{ + struct term *t = tt_new(80, 9); + assert(t); + const char *lines = "one\r\ntwo\r\nthree\r\nfour\r\nfive\r\nsix\r\nseven\r\n"; + tt_write(t, lines, strlen(lines)); + const char *rewrite = "\033[6Ftwo revised\033[K\033[2Efour revised\033[K\033[3Fone revised\033[K"; + tt_write(t, rewrite, strlen(rewrite)); + assert(row_is(t, 1, "two revised")); + assert(row_is(t, 3, "four revised")); + assert(row_is(t, 0, "one revised")); + term_free(t); +} + +static void test_relative_cursor_motion(void) +{ + struct term *t = tt_new(10, 5); + assert(t); + const char *text = "0123456789\r\nabcdefghij\r\nklmnopqrst"; + tt_write(t, text, strlen(text)); + const char *sequence = "\033[3`X\033[2aY\033[1eZ\033[2dW"; + tt_write(t, sequence, strlen(sequence)); + assert(tt_cell(t, 2, 2)->text == 'X'); + assert(tt_cell(t, 5, 2)->text == 'Y'); + assert(tt_cell(t, 6, 3)->text == 'Z'); + assert(tt_cell(t, 7, 1)->text == 'W'); + term_free(t); +} + +static void test_top_region_scrollback(void) +{ + static const struct { + const char *scroll; + int lines; + } cases[] = { + {"\n", 1}, + {"\033D", 1}, + {"\033E", 1}, + {"\033[S", 1}, + {"\033[2S", 2}, + {"\033[99S", 4}, + }; + for (size_t i = 0; i < sizeof(cases) / sizeof(cases[0]); i++) { + struct term *t = tt_new(12, 6); + write_seq(t, "one\r\ntwo\r\nthree\r\nfour\r\nPROMPT\r\nSTATUS"); + write_seq(t, "\033[1;4r\033[4;1H"); + write_seq(t, cases[i].scroll); + assert(tt_history(t) == cases[i].lines); + assert(row_is(t, 0, cases[i].lines == 1 ? "two" : + cases[i].lines == 2 ? "three" : "")); + assert(row_is(t, 3, "")); + assert(row_is(t, 4, "PROMPT") && row_is(t, 5, "STATUS")); + term_scroll(t, cases[i].lines); + assert(tt_view_offset(t) == cases[i].lines); + assert(row_is(t, 0, "one") && row_is(t, 1, "two")); + term_free(t); + } + + /* Retain the same history as a full screen after the ring wraps. */ + struct term *t = tt_new(12, 6); + struct term *reference = tt_new(12, 4); + const char *footer = "\033[5;1H\033[31mPROMPT\r\nSTATUS\033[m\033[1;4r"; + tt_write(t, footer, strlen(footer)); + for (int i = 0; i < 2 * TEST_SCROLLBACK; i++) { + char line[32]; + int len = snprintf(line, sizeof(line), "line %04d\r\n", i); + tt_write(t, line, (size_t)len); + tt_write(reference, line, (size_t)len); + } + assert(tt_history(t) == tt_history(reference)); + assert(tt_cursor_x(t) == tt_cursor_x(reference)); + assert(tt_cursor_y(t) == tt_cursor_y(reference)); + assert(row_is(t, 4, "PROMPT") && row_is(t, 5, "STATUS")); + assert(tt_cell(t, 0, 4)->fg == tt_cell(t, 0, 5)->fg); + assert(tt_cell(t, 0, 4)->fg != TERM_COLOR_FG); + for (int i = 0; i <= tt_history(reference); i++) { + assert_same_rows(t, reference, tt_rows(reference)); + term_scroll(t, 1); + term_scroll(reference, 1); + } + term_free(t); + term_free(reference); +} + +static void test_editor_scrolling(void) +{ + struct term *t = tt_new(80, 24); + write_seq(t, "\033[?1049hTITLE"); + for (int i = 1; i <= 20; i++) { + char line[64]; + snprintf(line, sizeof(line), "\033[%d;1Hline %02d", i + 1, i); + write_seq(t, line); + } + write_seq(t, "\033[23;1HHELP\033[24;1HFOOTER"); + + write_seq(t, "\033[4;22r\033[22;1H\n\033[1;24r" + "\033[21;1Hline 21\033[K\r\033[4d"); + assert(row_is(t, 0, "TITLE") && row_is(t, 2, "line 02")); + assert(row_is(t, 3, "line 04") && row_is(t, 19, "line 20")); + assert(row_is(t, 20, "line 21") && row_is(t, 21, "")); + assert(row_is(t, 22, "HELP") && row_is(t, 23, "FOOTER")); + assert(tt_cursor_x(t) == 0 && tt_cursor_y(t) == 3); + assert(tt_history(t) == 0); + term_free(t); + + t = tt_new(12, 6); + write_seq(t, "old\r\nHEAD\r\nA\r\nB\r\nC\r\nD\r\nFOOT"); + assert(tt_history(t) == 1); + write_seq(t, "\033[2;5r\033[3;2H\033[M"); + assert(row_is(t, 0, "HEAD") && row_is(t, 1, "A")); + assert(row_is(t, 2, "C") && row_is(t, 3, "D") && row_is(t, 4, "")); + assert(row_is(t, 5, "FOOT")); + /* DL moves the cursor to the left margin (xterm). */ + assert(tt_cursor_x(t) == 0 && tt_cursor_y(t) == 2); + write_seq(t, "\033[L"); + assert(row_is(t, 2, "") && row_is(t, 3, "C") && row_is(t, 4, "D")); + write_seq(t, "\033[2;1H\033M"); + assert(row_is(t, 1, "") && row_is(t, 2, "A") && row_is(t, 4, "C")); + write_seq(t, "\033[5;1H\033D"); + assert(row_is(t, 1, "A") && row_is(t, 3, "C") && row_is(t, 4, "")); + write_seq(t, "\033[T\033[S"); + assert(row_is(t, 1, "A") && row_is(t, 3, "C")); + + write_seq(t, "\033[5;2r\033[1;999999999999r\033[44m\033[999999999999S"); + assert(row_is(t, 0, "HEAD") && row_is(t, 5, "FOOT")); + for (int r = 1; r <= 4; r++) { + assert(row_is(t, r, "")); + assert(tt_cell(t, 0, r)->bg == tt_cell(t, 11, r)->bg); + assert(tt_cell(t, 0, r)->bg != TERM_COLOR_BG); + } + assert(tt_history(t) == 1); + + write_seq(t, "\033[6;1H\033[M\033[L"); + assert(row_is(t, 5, "FOOT")); + + tt_resize(t, 12, 4); + write_seq(t, "\033[4;1H\033[2Kend\n"); + assert(tt_cursor_y(t) == 3 && row_is(t, 2, "end")); + term_free(t); +} + +static bool pending_decsca_decsed_decsel(void) +{ + struct term *d = tt_new(20, 3); + write_seq(d, "\033[H" "AB" "\033[1\"q" "CD" "\033[0\"q" "EF"); + write_seq(d, "\033[?2J"); + bool ok = row_eq(d, 0, " CD"); + term_free(d); + + struct term *l = tt_new(20, 3); + write_seq(l, "\033[H" "GH" "\033[1\"q" "IJ" "\033[0\"q" "KL"); + write_seq(l, "\033[H\033[?0K"); + ok = ok && row_eq(l, 0, " IJ"); + term_free(l); + return ok; +} + +static bool pending_decfra_decera(void) +{ + struct term *f = tt_new(20, 3); + write_seq(f, "\033[88;1;2;2;4$x"); + bool ok = row_eq(f, 0, " XXX") && row_eq(f, 1, " XXX"); + term_free(f); + + struct term *e = tt_new(20, 3); + write_seq(e, "\033[Habcde\r\nfghij"); + write_seq(e, "\033[1;2;2;3$z"); + ok = ok && row_eq(e, 0, "a de") && row_eq(e, 1, "f ij"); + term_free(e); + return ok; +} + +static bool pending_deccra(void) +{ + struct term *t = tt_new(20, 3); + write_seq(t, "\033[HAB\r\nCD"); + write_seq(t, "\033[1;1;2;2;1;1;5;1$v"); + bool ok = row_eq(t, 0, "AB AB") && row_eq(t, 1, "CD CD"); + term_free(t); + return ok; +} + +static bool pending_decrara(void) +{ + struct term *t = tt_new(20, 3); + write_seq(t, "\033[H\033[1mAB\033[mCD"); + write_seq(t, "\033[1;1;1;4;1$t"); + bool ok = !(tt_cell(t, 0, 0)->attrs & TERM_ATTR_BOLD) && + (tt_cell(t, 2, 0)->attrs & TERM_ATTR_BOLD); + term_free(t); + return ok; +} + +static bool pending_decsera(void) +{ + struct term *t = tt_new(20, 3); + write_seq(t, "\033[H\033[0\"qAB\033[1\"qCD"); + write_seq(t, "\033[1;1;1;4${"); + bool ok = row_eq(t, 0, " CD"); + term_free(t); + return ok; +} + +static bool pending_deccolm(void) +{ + struct term *t = tt_new(80, 4); + write_seq(t, "\033[?40h" "keep" "\033[?3h"); + bool ok = tt_cols(t) == 132 && row_eq(t, 0, "") && + tt_cursor_x(t) == 0 && tt_cursor_y(t) == 0; + write_seq(t, "\033[?3l"); + ok = ok && tt_cols(t) == 80; + term_free(t); + return ok; +} + +static bool pending_decic_decdc(void) +{ + struct term *i = tt_new(10, 2); + write_seq(i, "\033[Habcdef\033[H\033[2'}"); + bool ok = row_eq(i, 0, " abcdef"); + term_free(i); + + struct term *d = tt_new(10, 2); + write_seq(d, "\033[Habcdef\033[H\033[2'~"); + ok = ok && row_eq(d, 0, "cdef"); + term_free(d); + return ok; +} + +static bool pending_sl_sr(void) +{ + struct term *l = tt_new(10, 2); + write_seq(l, "\033[Habcdef\033[2 @"); + bool ok = row_eq(l, 0, "cdef"); + term_free(l); + + struct term *r = tt_new(10, 2); + write_seq(r, "\033[Habcdef\033[2 A"); + ok = ok && row_eq(r, 0, " abcdef"); + term_free(r); + return ok; +} + +static bool pending_sixel(void) +{ + struct term *t = tt_new(20, 6); + write_seq(t, "\033[H\033Pq#0;2;100;100;100#0~~~~\033\\"); + bool ok = tt_cursor_y(t) > 0; + term_free(t); + return ok; +} + +static void test_unimplemented_da_features(void) +{ + static const struct { + const char *name; + bool (*run)(void); + } cases[] = { + { "DECSCA/DECSED/DECSEL selective erase (DA1 6)", pending_decsca_decsed_decsel }, + { "DECFRA/DECERA rectangle fill, erase (DA1 28)", pending_decfra_decera }, + { "DECCRA rectangle copy (DA1 28)", pending_deccra }, + { "DECRARA rectangle attributes (DA1 28)", pending_decrara }, + { "DECSERA rectangle selective erase (DA1 28)", pending_decsera }, + { "DECCOLM 132 columns (DA1 1)", pending_deccolm }, + { "DECIC/DECDC column insert, delete (DA1 21)", pending_decic_decdc }, + { "SL/SR scroll left, right (DA1 21)", pending_sl_sr }, + { "sixel graphics, cursor advance (DA1 4)", pending_sixel }, + }; + size_t total = sizeof(cases) / sizeof(cases[0]), ready = 0; + for (size_t i = 0; i < total; i++) { + if (!cases[i].run()) continue; + printf(" %s now passes: assert it and add its DA1 parameter\n", cases[i].name); + ready++; + } + printf("unimplemented features: %zu pending, %zu ready to promote\n", + total - ready, ready); +} + +static void test_device_attributes(void) +{ + struct term *t = tt_new_opts(80, 24, &reply_cb); + + write_seq(t, "\033[c"); + assert(!strcmp(response, "\033[?62;22c")); + response_len = 0; + write_seq(t, "\033[0c"); + assert(!strcmp(response, "\033[?62;22c")); + + write_seq(t, "\033[>c"); + assert(!strcmp(response, "\033[>0;10;0c")); + + response_len = 0; + write_seq(t, "\033[=c"); + assert(response_len == 0); + + term_free(t); +} + +static void expect_flags(struct term *t, const char *commands, const char *reply) +{ + write_seq(t, commands); + write_seq(t, "\033[?u"); + assert(!strcmp(response, reply)); +} +static void test_keyboard_negotiation(void) +{ + struct term *t = tt_new_opts(80, 24, &reply_cb); + assert(!term_modes(t).app_cursor); + write_seq(t, "\033[?1h"); + assert(term_modes(t).app_cursor); + write_seq(t, "\033[?1l"); + assert(!term_modes(t).app_cursor); + expect_flags(t, "", "\033[?0u"); + expect_flags(t, "\033[=1u", "\033[?1u"); + expect_flags(t, "\033[>3u\033[>31u", "\033[?31u"); + expect_flags(t, "\033[<u", "\033[?3u"); + expect_flags(t, "\033[=4;2u", "\033[?7u"); + expect_flags(t, "\033[=2;3u", "\033[?5u"); + expect_flags(t, "\033[=31;99u", "\033[?5u"); + expect_flags(t, "\033[<u", "\033[?1u"); + expect_flags(t, "\033[<999999999999999999999999u", "\033[?0u"); + expect_flags(t, "\033[>1:2u\033[=3;1;4u\033[1>31u", "\033[?0u"); + expect_flags(t, "\033[=255u", "\033[?31u"); + expect_flags(t, "\033[>u", "\033[?0u"); + expect_flags(t, "\033[<u", "\033[?31u"); + expect_flags(t, "\033[>1u\033[>2u\033[<2u", "\033[?31u"); + expect_flags(t, "\033[?1049h", "\033[?0u"); + expect_flags(t, "\033[>3u", "\033[?3u"); + expect_flags(t, "\033[?1049l", "\033[?31u"); + + for (int i = 0; i < 1000; i++) write_seq(t, "\033[>1u"); + expect_flags(t, "\033[<u", "\033[?1u"); + expect_flags(t, "\033[<2147483647u", "\033[?0u"); + write_seq(t, "hello"); + assert(row_is(t, 0, "hello")); + term_free(t); +} + +static void expect_selection(struct term *t, int x1, int y1, int x2, int y2, const char *want) +{ + tt_select(t, x1, y1, x2, y2); + char *text = tt_selection_text(t); + if (!text || strcmp(text, want)) + fprintf(stderr, "selection: got '%s', want '%s'\n", text ? text : "NULL", want); + assert(text && !strcmp(text, want)); + free(text); +} + +static void test_wide_characters(void) +{ + struct term *t = tt_new(8, 4); + write_seq(t, "A😀界B"); + assert(tt_cursor_x(t) == 6); + assert(tt_cell(t, 1, 0)->text == 0x1f600 && tt_cell(t, 3, 0)->text == 0x754c); + expect_selection(t, 2, 0, 2, 0, "😀"); + write_seq(t, "\033[3Gx\033[4GX"); + assert(row_is(t, 0, "A xX B")); + write_seq(t, "\033[2J\033[HA😀BC\033[2G\033[@"); + expect_selection(t, 0, 0, 7, 0, "A 😀BC"); + write_seq(t, "\033[3G\033[P"); + expect_selection(t, 0, 0, 7, 0, "A BC"); + write_seq(t, "\033[2J\033[HA😀BC\033[3G\033[X"); + assert(row_is(t, 0, "A BC")); + + write_seq(t, "\033[4;1H😀\r \033[4;1H😀\033[4;2H\033[38;2;1;2;3mX"); + assert(row_is(t, 3, " X")); + term_free(t); + + t = tt_new(5, 8); + write_seq(t, "abcd😀Z"); + assert(tt_cursor_y(t) == 1 && tt_cursor_x(t) == 3); + expect_selection(t, 0, 0, 4, 1, "abcd😀Z"); + tt_resize(t, 3, 8); + assert(tt_cell(t, 1, 1)->text == 0x1f600 && tt_cell(t, 0, 2)->text == 'Z'); + assert(tt_cursor_y(t) == 2 && tt_cursor_x(t) == 1); + expect_selection(t, 0, 0, 2, 2, "abcd😀Z"); + tt_resize(t, 12, 8); + assert(tt_cursor_x(t) == 7 && tt_cursor_y(t) == 0); + expect_selection(t, 0, 0, 11, 0, "abcd😀Z"); + tt_resize(t, 1, 8); + tt_resize(t, 12, 8); + expect_selection(t, 0, 0, 11, 0, "abcd😀Z"); + assert(tt_cursor_x(t) == 7); + term_free(t); + + t = tt_new(4, 3); + write_seq(t, "abc©️)"); + assert(tt_cursor_y(t) == 1 && tt_cursor_x(t) == 3); + expect_selection(t, 0, 0, 3, 1, "abc©️)"); + write_seq(t, "\033[2J\033[H😀︎x"); + assert(tt_cursor_x(t) == 2 && tt_cell(t, 1, 0)->text == 'x'); + expect_selection(t, 0, 0, 3, 0, "😀︎x"); + write_seq(t, "\033[2J\033[H\033[?7labc😀"); + assert(tt_cursor_y(t) == 0 && tt_cursor_x(t) == 3); + term_free(t); + + t = tt_new(4, 2); + write_seq(t, "😀😀界界AB"); + term_scroll(t, 1); + expect_selection(t, 0, 0, 3, 1, "😀😀界界"); + term_scroll_to(t, 0); + tt_resize(t, 10, 2); + expect_selection(t, 0, 0, 9, 0, "😀😀界界AB"); + term_free(t); +} + +static void test_wide_runs(void) +{ + static const char *const runs[] = { + "日本語漢字仮名交じり文中文測試終端輸出性能比較日本語漢字仮名交じり文", + " 日本語漢字仮名交じり文中文測試終端輸出性能比較", + "┌─────────────────────┬─────────────────────┐", + "日本語漢字仮名交じり文abc中文測試終端輸出性能比較", + "日本語漢字仮名交じり文̈中文測試終端輸出性能比較", + "한국어를테스트하는줄을적습니다한국어를테스트하는줄을적습니다", + "！＂＃＄％＆＇（）＊＋，－．／０１２３４５６７８９", + "日本語漢字仮名交じり文😀中文測試終端輸出性能比較", + }; + for (size_t k = 0; k < sizeof(runs) / sizeof(*runs); k++) { + char text[512]; + snprintf(text, sizeof(text), "\033[999;1H%s", runs[k]); + struct term *bulk = tt_new(120, 6); + struct term *split = tt_new(120, 6); + tt_write(bulk, text, strlen(text)); + write_seq(split, text); + assert_same_screen(bulk, split); + + expect_selection(bulk, 0, 5, 119, 5, runs[k]); + term_free(bulk); + term_free(split); + } +} + +static void test_invalid_utf8_runs(void) +{ + static const unsigned char invalid[][3] = { + {0xe0, 0x82, 0xa1}, {0xed, 0xa0, 0x80}, {0xe0, 0x9f, 0xbf}, + }; + for (size_t k = 0; k < sizeof(invalid) / sizeof(*invalid); k++) { + char input[128]; + size_t n = 0; + for (int i = 0; i < 32; i++) { + memcpy(input + n, invalid[k], 3); + n += 3; + } + memcpy(input + n, "END", 3); + n += 3; + struct term *bulk = tt_new(120, 4); + struct term *split = tt_new(120, 4); + write_seq(bulk, "\033[4;1H\n"); + write_seq(split, "\033[4;1H\n"); + tt_write(bulk, input, n); + for (size_t i = 0; i < n; i++) tt_write(split, input + i, 1); + /* Each byte is a maximal invalid subpart: one U+FFFD each. */ + for (int c = 0; c < 96; c++) + assert(tt_cell_cp(bulk, tt_cell(bulk, c, 3)) == 0xFFFD); + for (int c = 0; c < 3; c++) + assert(tt_cell_cp(bulk, tt_cell(bulk, 96 + c, 3)) == (uint32_t)"END"[c]); + assert_same_screen(bulk, split); + term_free(bulk); + term_free(split); + } +} + +static void test_recycled_scrollback_blanks(void) +{ + for (int mode = 0; mode < 3; mode++) { + struct term *t = tt_new(32, 3); + write_seq(t, "\033[3;1H"); + const char *line = mode == 2 ? "abcdefghijklmnopqrstuvwxyz\r\033[S" : + "abcdefghijklmnopqrstuvwxyz\r\n"; + for (int i = 0; i < TEST_SCROLLBACK + 10; i++) { + if (mode == 1) write_seq(t, line); + else tt_write(t, line, strlen(line)); + } + write_seq(t, "\tfile/"); + assert(row_is(t, 2, " file/")); + term_free(t); + } +} + +/* Scrolling the view by any amount lands inside the scrollback, and a + * scroll that cannot move leaves the frame alone. */ +static void test_scroll_view_clamps_extreme_deltas(void) +{ + struct term *t = tt_new(20, 3); + for (int i = 0; i < 10; i++) + write_seq(t, "line\r\n"); + int max = tt_history(t); + assert(max > 1); + struct term_snapshot *snap = term_snapshot_new(); + term_scroll(t, 1); + term_scroll(t, INT_MAX); + assert(tt_view_offset(t) == max); + uint64_t version = view_version(snap, t); + term_scroll(t, 1); + assert(view_version(snap, t) == version); + term_scroll(t, -1); + term_scroll(t, INT_MIN); + assert(tt_view_offset(t) == 0); + version = view_version(snap, t); + term_scroll(t, -1); + assert(view_version(snap, t) == version); + term_snapshot_free(snap); + term_free(t); +} + +static void test_batched_recycled_lines(void) +{ + static const char *const endings[] = { + "short\r\nx\r\nyz\r\n", + "short\r\nx\r\nyz\r\n\tfile/", + "abcdefghijklmnopqrstuvwxyz012345more\r\nx\r\nyz\r\n\033[2CZ", + "short\r\nx\r\nyz\r\n界\xcc\x81", + "short\r\nx\r\nyz\r\npartial\r", + }; + for (size_t k = 0; k < sizeof(endings) / sizeof(endings[0]); k++) { + struct term *bulk = tt_new(32, 3); + struct term *bytes = tt_new(32, 3); + const char *old = "\033[31;44mabcdefghijklmnopqrst界界界\r\n"; + for (int i = 0; i < TEST_SCROLLBACK + 10; i++) { + tt_write(bulk, old, strlen(old)); + write_seq(bytes, old); + } + write_seq(bulk, "\033[0;1;3m"); + write_seq(bytes, "\033[0;1;3m"); + tt_write(bulk, endings[k], strlen(endings[k])); + write_seq(bytes, endings[k]); + + assert_same_screen(bulk, bytes); + for (int i = 0; i < tt_history(bulk); i += 3) { + term_scroll(bulk, 3); + term_scroll(bytes, 3); + assert_same_screen(bulk, bytes); + } + term_free(bulk); + term_free(bytes); + } +} + +static void test_full_width_line_endings(void) +{ + static const char *const feeds[] = { + "aaaa\r\nbbbb\r\ncccc\r\ndddd\r\neeee\r\nffff", + "aaaa\r\nbbbb\r\ncccc\r\ndddd\neeee", + "aaaa\r\nbbbb\r\ncccc\r\ndddd\re", + "aaaa\r\nbbbb\r\ncccc\r\nddddeeeeff", + "aa\r\nbb\r\ncc\r\nd中d\r\nef", + }; + for (size_t k = 0; k < sizeof(feeds) / sizeof(feeds[0]); k++) { + struct term *bulk = tt_new(4, 3); + struct term *bytes = tt_new(4, 3); + tt_write(bulk, feeds[k], strlen(feeds[k])); + write_seq(bytes, feeds[k]); + assert_same_screen(bulk, bytes); + term_free(bulk); + term_free(bytes); + } + + struct term *t = tt_new(4, 3); + tt_write(t, feeds[0], strlen(feeds[0])); + assert(tt_history(t) == 3); + assert(row_is(t, 0, "dddd") && row_is(t, 1, "eeee") && row_is(t, 2, "ffff")); + term_free(t); +} + +/* Compare the screen, cursor, and scrollback for each input mode. + * + * Input mode | Byte-wise | 64 KiB bulk | 512 KiB single write + * Raw LF | Reference | Must match | Must match + * Newline conversion | Reference | Must match | Must match + * + * The input length selects the stream path. The 512 KiB write also has + * enough newlines to test the backward scan when newline conversion is + * active. + */ +static void test_bulk_line_flood(void) +{ + static const struct term_vector2i geoms[] = { + {.x = 80, .y = 24}, {.x = 27, .y = 8}, {.x = 10, .y = 5}, {.x = 120, .y = 80}, + }; + for (size_t g = 0; g < 2 * sizeof(geoms) / sizeof(geoms[0]); g++) { + int cols = geoms[g / 2].x, rows = geoms[g / 2].y; + size_t cap = 1u << 19; + char *buf = malloc(cap); + assert(buf); + size_t n = 0; + unsigned state = 12345; + while (n < cap / 2) { + state = state * 1103515245u + 12345u; + unsigned pick = (state >> 16) % 27; + buf[n++] = pick == 26 ? '\n' : (char)('a' + pick); + } + for (int k = -1; k <= 1; k++) { + memset(buf + n, 'x', (size_t)(cols + k)); + n += (size_t)(cols + k); + buf[n++] = '\n'; + memset(buf + n, 'y', (size_t)(2 * cols + k)); + n += (size_t)(2 * cols + k); + buf[n++] = '\n'; + } + while (n < cap) { + state = state * 1103515245u + 12345u; + unsigned pick = (state >> 16) % 27; + buf[n++] = pick == 26 ? '\n' : (char)('a' + pick); + } + + struct term *bulk = tt_new(cols, rows); + struct term *stream = tt_new(cols, rows); + struct term *bytes = tt_new(cols, rows); + unsigned flags = g % 2 != 0 ? TERM_FEED_ONLCR : 0; + for (size_t i = 0; i < n; i += 65536) + term_feed(bulk, buf + i, n - i < 65536 ? n - i : 65536, flags); + /* One large write has enough newlines to discard old output. */ + term_feed(stream, buf, n, flags); + for (size_t i = 0; i < n; i++) term_feed(bytes, buf + i, 1, flags); + + assert_same_screen(bulk, bytes); + assert_same_screen(stream, bytes); + for (int i = 0; i < tt_history(bulk); i += 7) { + term_scroll(bulk, 7); + term_scroll(stream, 7); + term_scroll(bytes, 7); + assert_same_screen(bulk, bytes); + assert_same_screen(stream, bytes); + } + term_free(bulk); + term_free(stream); + term_free(bytes); + free(buf); + } +} + +/* Dense LF input must keep its text through edits and buffer moves. */ +static void test_ascii_flood_edits(void) +{ + static const int widths[] = {1, 2, 7, 120}; + char input[65536]; + unsigned state = 43; + for (size_t i = 0; i < sizeof(input); i++) { + state = state * 1103515245u + 12345u; + unsigned pick = (state >> 16) % 5; + input[i] = pick == 4 ? '\n' : (char)('a' + pick); + } + memset(input + 1024, '\n', 256); + static const char edit[] = + "\033[1;1Hé漢x̂\033[T\033[2;1HEND" + "\033[?1049hALT\033[?1049l\033[6;1Htail\n"; + for (size_t w = 0; w < sizeof(widths) / sizeof(widths[0]); w++) { + for (size_t chunk = 4096; chunk <= sizeof(input); chunk *= 16) { + struct term *bulk = tt_new(widths[w], 6); + struct term *bytes = tt_new(widths[w], 6); + for (size_t i = 0; i < sizeof(input); i += chunk) + term_feed(bulk, input + i, chunk, TERM_FEED_ONLCR); + for (size_t i = 0; i < sizeof(input); i++) + term_feed(bytes, input + i, 1, TERM_FEED_ONLCR); + assert_same_screen(bulk, bytes); + term_feed(bulk, edit, sizeof(edit) - 1, TERM_FEED_ONLCR); + for (size_t i = 0; i < sizeof(edit) - 1; i++) + term_feed(bytes, edit + i, 1, TERM_FEED_ONLCR); + assert_same_screen(bulk, bytes); + for (int i = 0; i < tt_history(bulk); i += 6) { + term_scroll(bulk, 6); + term_scroll(bytes, 6); + assert_same_screen(bulk, bytes); + } + term_scroll_to(bulk, 0); + term_scroll_to(bytes, 0); + assert(tt_resize(bulk, widths[w] + 3, 7)); + assert(tt_resize(bytes, widths[w] + 3, 7)); + assert_same_screen(bulk, bytes); + term_free(bulk); + term_free(bytes); + } + } +} + +/* Newline modes: 0 is a plain line feed, 1 is LNM (CSI 20 h), and 2 is + * newline conversion on the feed. */ +static void test_margin_linefeed_bursts(void) +{ + static const char *bursts[] = { + "\n\n", "\n\n\n\n\n", "\n\n\n\n\n\n\n\n", + "\r\n\r\n", "\r\n\r\n\r\n\r\n\r\n\r\n", + "\n\n\r\n\r\n\n\n", + }; + for (size_t k = 0; k < sizeof(bursts) / sizeof(bursts[0]); k++) { + for (int mode = 0; mode < 3; mode++) { + struct term *bulk = tt_new(8, 8); + struct term *bytes = tt_new(8, 8); + const char *setup = "\033[1;1Htop\033[2;1Ha\033[3;1Hb\033[4;1Hc" + "\033[5;1Hd\033[6;1He\033[7;1Hf\033[8;1Hfoot" + "\033[2;7r\033[7;3H"; + tt_write(bulk, setup, strlen(setup)); + tt_write(bytes, setup, strlen(setup)); + if (mode == 1) { + write_seq(bulk, "\033[20h"); + write_seq(bytes, "\033[20h"); + } + unsigned flags = mode == 2 ? TERM_FEED_ONLCR : 0; + term_feed(bulk, bursts[k], strlen(bursts[k]), flags); + feed_seq(bytes, bursts[k], flags); + assert_same_screen(bulk, bytes); + assert(row_is(bulk, 0, "top")); + assert(row_is(bulk, 7, "foot")); + term_free(bulk); + term_free(bytes); + } + } +} + +static void test_alt_linefeed_burst_after_resize(void) +{ + struct term *bulk = tt_new(8, 5); + struct term *bytes = tt_new(8, 5); + const char *setup = "\033[?1049h\033[5;1H\n\n\n\n"; + tt_write(bulk, setup, strlen(setup)); + write_seq(bytes, setup); + tt_resize(bulk, 8, 3); + tt_resize(bytes, 8, 3); + const char *burst = "\n\n\n\n\r\n\r\n"; + tt_write(bulk, burst, strlen(burst)); + write_seq(bytes, burst); + assert_same_screen(bulk, bytes); + tt_resize(bulk, 12, 4); + tt_resize(bytes, 12, 4); + assert_same_screen(bulk, bytes); + term_free(bulk); + term_free(bytes); +} + +/* + * Regression: the alternate screen scrolls by moving its top row. A resize + * changed the row count but kept that top row, so rows came back out of + * order, and further line feeds indexed past the row tables. + */ +static void test_alt_resize_keeps_row_order(void) +{ + const char *setup = "\033[?1049h\033[5;1H\n\n\n" + "\033[1;1HA\033[2;1HB\033[3;1HC\033[4;1HD\033[5;1HE"; + + struct term *grow = tt_new(20, 5); + write_seq(grow, setup); + tt_resize(grow, 20, 8); + assert(row_is(grow, 0, "A") && row_is(grow, 4, "E")); + assert(row_is(grow, 5, "") && row_is(grow, 7, "")); + term_free(grow); + + /* The cursor is on the last row, so shrinking scrolls A and B off. */ + struct term *shrink = tt_new(20, 5); + write_seq(shrink, setup); + tt_resize(shrink, 20, 3); + assert(row_is(shrink, 0, "C") && row_is(shrink, 2, "E")); + term_free(shrink); +} + +static void test_alt_shrink_then_flood(void) +{ + struct term *t = tt_new(80, 40); + tt_write(t, "\033[?1049h\033[40;1H", 16); + for (int i = 0; i < 35; i++) + tt_write(t, "x\n", 2); + tt_resize(t, 80, 10); + /* More lines than the ring holds, one write per line. */ + char line[32]; + for (int i = 0; i < 6000; i++) { + int len = snprintf(line, sizeof(line), "\rline %04d\n", i); + tt_write(t, line, (size_t)len); + } + assert(row_is(t, 8, "line 5999") && row_is(t, 0, "line 5991")); + term_free(t); +} + +/* Regression: ED 3 on the alternate screen kept the history count, so the + * normal screen came back with blank history rows. */ +static void test_alt_erase_scrollback(void) +{ + struct term *t = tt_new(20, 5); + for (int i = 0; i < 30; i++) + tt_write(t, "line\r\n", 6); + assert(tt_history(t) > 0); + write_seq(t, "\033[?1049h\033[3J\033[?1049l"); + assert(tt_history(t) == 0); + term_free(t); +} + +/* Regression: a snapshot knew its terminal by address. A new terminal that + * reused a freed one's memory then showed the old terminal's rows. */ +static void test_snapshot_follows_new_terminal(void) +{ + struct term_snapshot *s = term_snapshot_new(); + for (int i = 0; i < 8; i++) { + struct term *t = tt_new(10, 2); + char text[2] = { (char)('a' + i), 0 }; + tt_puts(t, text); + assert(term_snapshot_update(s, t)); + assert(term_snapshot_row(s, 0)[0].text == (uint32_t)text[0]); + term_free(t); + } + term_snapshot_free(s); +} + +static void test_string_payloads(void) +{ + static const char *prefixes[] = {"\033]0;", "\033]7;file://host/", "\033P"}; + static const char *ends[] = {"\007", "\033\\"}; + static const size_t chunks[] = {1, 7, 4096}; + for (size_t p = 0; p < 3; p++) { + for (size_t e = 0; e < 2; e++) { + for (int large = 0; large < 2; large++) { + char input[1024]; + size_t n = (size_t)snprintf(input, sizeof(input), "%s⠋ çalışma/clernen_pi", prefixes[p]); + if (large) { + memset(input + n, 'a', 600); + n += 600; + } + n += (size_t)snprintf(input + n, sizeof(input) - n, "%sOK", ends[e]); + for (size_t c = 0; c < 3; c++) { + title[0] = 0; + response_len = 0; + struct term *t = tt_new_opts(80, 4, &title_cb); + for (size_t i = 0; i < n;) { + size_t take = n - i < chunks[c] ? n - i : chunks[c]; + tt_write(t, input + i, take); + i += take; + } + assert(row_is(t, 0, "OK")); + assert(tt_cursor_x(t) == 2 && tt_cursor_y(t) == 0); + assert(response_len == 0); + if (p == 0 && !large) + assert(!strcmp(title, "⠋ çalışma/clernen_pi")); + if (large) { + if (p == 0) { + assert(!strncmp(title, "⠋ çalışma/clernen_pi", + strlen("⠋ çalışma/clernen_pi"))); + assert(title[sizeof(title) - 1] == 0); + } else { + assert(title[0] == 0); + } + } + term_free(t); + } + } + } + } +} + +static void test_newline_policy_survives_reset(void) +{ + static const char *const resets[] = { "\033c", "\033[!p", "\033[20l" }; + for (size_t i = 0; i < sizeof(resets) / sizeof(resets[0]); i++) { + struct term *t = tt_new(20, 6); + term_feed(t, "a\n", 2, TERM_FEED_ONLCR); + assert(tt_cursor_x(t) == 0); + write_seq(t, resets[i]); + term_feed(t, "b\nc", 3, TERM_FEED_ONLCR); + assert(tt_cursor_x(t) == 1); + assert(row_is(t, tt_cursor_y(t), "c")); + term_free(t); + } +} + +static void test_lnm_mode_alone_is_not_sticky(void) +{ + struct term *t = tt_new(20, 6); + write_seq(t, "\033[20h\033c"); + tt_write(t, "b\nc", 3); + assert(tt_cursor_x(t) == 2); + term_free(t); +} + +/* A C0 control ends an unfinished UTF-8 sequence, which prints U+FFFD. A + * lone continuation byte prints U+FFFD too. */ +static void test_c0_ends_utf8_sequence(void) +{ + static const char input[] = "\xc3\n\xa9z"; + for (size_t split = 0; split < sizeof(input) - 1; split++) { + struct term *t = tt_new(20, 4); + tt_write(t, input, split); + tt_write(t, input + split, sizeof(input) - 1 - split); + assert(tt_cell_cp(t, tt_cell(t, 0, 0)) == 0xFFFD); + assert(tt_cell_cp(t, tt_cell(t, 1, 1)) == 0xFFFD); + assert(tt_cell_cp(t, tt_cell(t, 2, 1)) == 'z'); + term_free(t); + } +} + +/* A bulk stream of long lines must behave like byte-wise input after every + * later operation that touches those rows: overwriting, scroll-region + * rotation, reflow, alternate-screen round trips, and selection. */ +static void test_bulk_rows_survive_edits(void) +{ + enum { COLS = 40, ROWS = 6 }; + static char flood[1 << 16]; + for (size_t i = 0; i < sizeof(flood); i++) + flood[i] = (char)('a' + (i * 7 + i / COLS) % 26); + + struct term *bulk = tt_new(COLS, ROWS); + struct term *bytes = tt_new(COLS, ROWS); + write_seq(bulk, "\033[1;4;38;2;9;8;7;48;2;1;2;3m"); + write_seq(bytes, "\033[1;4;38;2;9;8;7;48;2;1;2;3m"); + tt_write(bulk, flood, sizeof(flood)); + for (size_t i = 0; i < sizeof(flood); i++) tt_write(bytes, flood + i, 1); + assert_same_screen(bulk, bytes); + + /* Overwrite the middle of a streamed row, then erase part of another. */ + const char *edit = "\033[m\033[2;5HXYZ\033[3;1H\033[K\033[4;10H\033[1K"; + write_seq(bulk, edit); + write_seq(bytes, edit); + assert_same_screen(bulk, bytes); + assert(tt_view_cell(bulk, 4, 1)->text == 'X'); + + /* Scroll region rotation moves row pointers between ring slots. */ + const char *region = "\033[2;5r\033[5;1H\n\n\033[T\033[r"; + write_seq(bulk, region); + write_seq(bytes, region); + assert_same_screen(bulk, bytes); + + /* Alternate screen and back, then view history. */ + write_seq(bulk, "\033[?1049hALT\033[?1049l"); + write_seq(bytes, "\033[?1049hALT\033[?1049l"); + assert_same_screen(bulk, bytes); + for (int i = 0; i < 3; i++) { + term_scroll(bulk, 100); + term_scroll(bytes, 100); + assert_same_screen(bulk, bytes); + } + term_scroll_to(bulk, 0); + term_scroll_to(bytes, 0); + + /* Selection text reads rows straight from history. */ + term_scroll(bulk, 50); + term_scroll(bytes, 50); + tt_select(bulk, 0, 0, COLS - 1, 2); + tt_select(bytes, 0, 0, COLS - 1, 2); + char *a = tt_selection_text(bulk), *b = tt_selection_text(bytes); + assert(a && b && !strcmp(a, b) && strlen(a) >= COLS); + free(a); + free(b); + term_scroll_to(bulk, 0); + term_scroll_to(bytes, 0); + + /* Reflow narrower and wider keeps history identical. */ + tt_resize(bulk, 29, ROWS); + tt_resize(bytes, 29, ROWS); + assert_same_screen(bulk, bytes); + tt_resize(bulk, 61, ROWS + 2); + tt_resize(bytes, 61, ROWS + 2); + assert_same_screen(bulk, bytes); + term_scroll(bulk, 20); + term_scroll(bytes, 20); + assert_same_screen(bulk, bytes); + + term_free(bulk); + term_free(bytes); +} + +static void test_erase_keeps_used_rows(void) +{ + for (int row = 0; row < 40; row++) { + struct term *t = tt_new(20, 40); + char seq[64]; + int len = snprintf(seq, sizeof(seq), "\033[%d;1Hsaved\033[2J", row + 1); + tt_write(t, seq, (size_t)len); + assert(tt_history(t) == row + 1); + for (int y = 0; y < 40; y++) + assert(row_eq(t, y, "")); + term_scroll(t, 1); + assert(row_eq(t, 0, "saved")); + term_free(t); + } +} + +static void test_blank_rows(void) +{ + /* ED 2 pushes used rows to scrollback; the erased screen reads as + * default-color blanks through every accessor. */ + struct term *t = tt_new(10, 4); + write_seq(t, "one\r\ntwo\r\nthree"); + write_seq(t, "\033[2J"); + assert(tt_history(t) == 3); + for (int y = 0; y < 4; y++) + for (int x = 0; x < 10; x++) { + const struct term_cell *c = tt_view_cell(t, x, y); + assert(c->text == ' ' && c->fg == TERM_COLOR_FG && + c->bg == TERM_COLOR_BG); + assert(tt_cell(t, x, y)->text == ' '); + } + term_scroll(t, 3); + assert(row_is(t, 0, "one") && row_is(t, 1, "two") && row_is(t, 2, "three")); + term_scroll_to(t, 0); + + /* A scrolled-in blank row reads as blanks, then accepts new text. */ + write_seq(t, "\033[4;1Hlast\r\n\r\n\r\n"); + for (int x = 0; x < 10; x++) + assert(tt_view_cell(t, x, 3)->text == ' '); + write_seq(t, "\033[4;6Hhi"); + assert(row_is(t, 3, " hi")); + term_free(t); + + /* A spaces-only row does not extend ED 2 history. */ + t = tt_new(10, 6); + write_seq(t, "top\033[6;1H "); + assert(tt_history(t) == 0); + write_seq(t, "\033[2J"); + assert(tt_history(t) == 1); + term_scroll(t, 1); + assert(row_is(t, 0, "top")); + term_scroll_to(t, 0); + + /* Blank rows survive a resize and still read as blanks. */ + tt_resize(t, 20, 6); + for (int x = 0; x < 20; x++) + assert(tt_cell(t, x, 5)->text == ' ' && + tt_cell(t, x, 5)->bg == TERM_COLOR_BG); + term_free(t); + + /* Erasing a blank row with a color still produces a colored row. */ + t = tt_new(10, 4); + write_seq(t, "\033[2J\033[41m\033[2K"); + assert(tt_cell(t, 0, 0)->bg != TERM_COLOR_BG); + assert(tt_cell(t, 9, 0)->bg == tt_cell(t, 0, 0)->bg); + term_free(t); +} + +static void test_csi_text_chunks(void) +{ + static const char *cases[] = { + "\033[2;3Hhello\033[2DXY\033[K\033[1A\033[3C!\r\n", + "\033[H日本語\033[2GXY\033[7mabc\033[m!\033[2Knext", + "\033[Hscreen\033[2J\033[Hnext\033[2K\r\n", + "\033[H012345678901234567890123\033[2DXY\033[J!", + "\033[?7l\033[H012345678901234567890123\033[1CXY", + "\033[41m\033[2J\033[Htext\033[0Jtail\033[1Jend", + "\033[2;3Htext\033[3Jnext\033[0K\033[1K\033[2K!", + "\033[?6h\033[2;3Htext\033[Jnext", + "\033[16;1Ha\033[17;1Hb\033[32;1Hc\033[40;1Hd\033[2J", + "\033[40;1H \033[17;1Ha\033[2J\033[Hnext", + "\033[?1049h\033[40;1H\n\033[17;1Htext\033[2J", + }; + for (size_t c = 0; c < sizeof(cases) / sizeof(cases[0]); c++) { + const char *seq = cases[c]; + size_t len = strlen(seq); + struct term *expected = tt_new(20, 40); + write_seq(expected, seq); + for (size_t split = 0; split <= len; split++) { + struct term *actual = tt_new(20, 40); + tt_write(actual, seq, split); + tt_write(actual, seq + split, len - split); + assert(tt_cursor_x(actual) == tt_cursor_x(expected)); + assert(tt_cursor_y(actual) == tt_cursor_y(expected)); + assert(tt_history(actual) == tt_history(expected)); + int history = tt_history(expected); + term_scroll(actual, history); + term_scroll(expected, history); + for (int page = 0; page <= history; page++) { + assert_same_rows(actual, expected, 40); + term_scroll(actual, -1); + term_scroll(expected, -1); + } + term_free(actual); + } + term_free(expected); + } +} + +/* A terminal grows past its first reservation: taller than 256 rows and + * wider than 2048 columns, on both screens. History and the saved screen + * keep their text. */ +static void test_grow_past_reservation(void) +{ + struct term *t = tt_new(40, 10); + for (int i = 0; i < 30; i++) { + char line[32]; + int len = snprintf(line, sizeof(line), "hist %02d\r\n", i); + tt_write(t, line, (size_t)len); + } + int history = tt_history(t); + assert(history > 0); + + tt_resize(t, 40, 700); + assert(tt_rows(t) == 700); + tt_write(t, "\033[700;1Hbottom", 14); + assert(row_is(t, 699, "bottom")); + + tt_resize(t, 3000, 700); + assert(tt_cols(t) == 3000); + tt_write(t, "\033[698;2995Hedge!", 16); + assert(tt_cell(t, 2994, 697)->text == 'e' && tt_cell(t, 2998, 697)->text == '!'); + + tt_write(t, "\033[700;1H", 8); + for (int i = 0; i < 1000; i++) { + char line[32]; + int len = snprintf(line, sizeof(line), "\r\nfill %04d", i); + tt_write(t, line, (size_t)len); + } + int history_before = tt_history(t); + assert(history_before > 0); + + tt_write(t, "\033[?1049h", 8); + tt_write(t, "alt", 3); + tt_resize(t, 3000, 2500); + assert(tt_rows(t) == 2500); + assert(row_is(t, 0, "alt")); + tt_write(t, "\033[?1049l", 8); + + bool found = false; + term_scroll(t, tt_history(t)); + for (int y = 0; y < tt_rows(t) && !found; y++) + found = row_eq(t, y, "hist 00"); + assert(found); + term_scroll(t, -tt_history(t)); + bool last = false; + for (int y = 0; y < tt_rows(t) && !last; y++) + last = row_eq(t, y, "fill 0999"); + assert(last); + term_free(t); +} + +struct concurrent_worker_args { + struct term *t; + _Atomic bool done; +}; + +static void *concurrent_feed_worker(void *arg) +{ + struct concurrent_worker_args *w = arg; + static const char stream[] = + "The quick brown fox jumps over the lazy dog.\r\n" + "\033[31mRed text\033[0m and \033[32mgreen text\033[0m.\r\n" + "Some unicode: 漢字, ひらがな, and emojis: 😀🚀🎉\r\n" + "\033[2J\033[HCleared and reset\r\n" + "Line 1\r\nLine 2\r\nLine 3\r\nLine 4\r\nLine 5\r\n" + "\033]7;file:///tmp/dir\007\033]22;pointer\007" + "\033]8;;https://example.com\007link\033]8;;\007\r\n"; + size_t len = strlen(stream); + + for (int i = 0; i < 2000; i++) { + term_feed(w->t, stream, len, 0); + } + atomic_store(&w->done, true); + return NULL; +} + +static void test_concurrent_feed_and_snapshot(void) +{ + struct term *t = term_new(&(struct term_options){ + .size = { .x = 80, .y = 24 }, + .scrollback = 1000, + }); + assert(t != NULL); + + struct concurrent_worker_args w = { .t = t, .done = false }; + pthread_t th; + int err = pthread_create(&th, NULL, concurrent_feed_worker, &w); + assert(err == 0); + + struct term_snapshot *snap = term_snapshot_new(); + assert(snap != NULL); + + while (!atomic_load(&w.done)) { + bool ok = term_snapshot_update(snap, t); + assert(ok); + struct term_vector2i sz = term_size(t); + assert(sz.x > 0 && sz.y > 0); + term_scroll(t, 2); + term_scroll_to(t, 0); + struct term_modes m = term_modes(t); + (void)m; + struct term_cursor c = term_cursor(t); + (void)c; + + /* Every reader is safe while another thread feeds. */ + char buf[256]; + term_select_start(t, (struct term_vector2i){ .x = 0, .y = 0 }, false, TERM_SELECT_WORD); + term_select_extend(t, (struct term_vector2i){ .x = 40, .y = 10 }, false); + (void)term_has_selection(t); + (void)term_selection_text(t, buf, sizeof(buf)); + (void)term_link_at(t, (struct term_vector2i){ .x = 1, .y = 1 }, buf, sizeof(buf)); + (void)term_cwd(t, buf, sizeof(buf)); + (void)term_pointer_shape(t, buf, sizeof(buf)); + (void)term_history_rows(t); + (void)term_history_text(t, 5, buf, sizeof(buf)); + struct term_scrollback sb = term_scrollback(t); + assert(sb.offset >= 0 && sb.total >= 0); + } + + pthread_join(th, NULL); + bool ok = term_snapshot_update(snap, t); + assert(ok); + + term_snapshot_free(snap); + term_free(t); +} + +int main(void) +{ + test_concurrent_feed_and_snapshot(); + test_grow_past_reservation(); + test_skipped_cells_read_blank(); + test_csi_text_chunks(); + test_erase_keeps_used_rows(); + test_blank_rows(); + test_lnm_mode_alone_is_not_sticky(); + test_newline_policy_survives_reset(); + test_esc_control_strings_and_designators(); + test_string_payloads(); + test_recycled_scrollback_blanks(); + test_scroll_view_clamps_extreme_deltas(); + test_batched_recycled_lines(); + test_invalid_utf8_runs(); + test_c0_ends_utf8_sequence(); + test_full_width_line_endings(); + test_bulk_line_flood(); + test_ascii_flood_edits(); + test_margin_linefeed_bursts(); + test_alt_linefeed_burst_after_resize(); + test_alt_resize_keeps_row_order(); + test_alt_shrink_then_flood(); + test_alt_erase_scrollback(); + test_snapshot_follows_new_terminal(); + test_bulk_rows_survive_edits(); + + assert(!strcmp(UNICODE_DATA_VERSION, "18.0.0")); + assert(!strcmp(term_unicode_version(), UNICODE_DATA_VERSION)); + assert(unicode_width('A') == 1); + assert(unicode_width(0x0301) == 0); + assert(unicode_width(0x754C) == 2); + assert(unicode_width(0x1FAE0) == 2); + assert(unicode_emoji_presentation(0x1FAE0)); + assert(unicode_width(0x1FAEB) == 2); + assert(unicode_emoji_presentation(0x1FAEB)); + { + /* A Jamo Extended-B vowel draws inside its leading consonant. */ + struct term *t = term_new(&(struct term_options){ .size = { .x = 10, .y = 2 } }); + term_feed(t, "\xe1\x84\x80\xed\x9e\xb0", 6, 0); + assert(term_cursor(t).pos.x == 2); + term_free(t); + } + + test_wide_characters(); + test_wide_runs(); + test_editor_scrolling(); + test_top_region_scrollback(); + test_device_attributes(); + test_unimplemented_da_features(); + test_keyboard_negotiation(); + test_resize_reflow(); + test_resize_exposes_blank_rows(); + test_prompt_resize(); + test_cursor_line_motion(); + test_relative_cursor_motion(); + test_read_boundaries(); + test_ascii_controls(); + test_random_read_boundaries(); + test_csi_recovery(); + test_csi_byte_boundaries(); + test_parameter_limits(); + test_unterminated_control_strings_do_not_swallow_text(); + test_c0_controls_after_esc_are_executed(); + struct term *t = tt_new_opts(10, 4, &reply_cb); + write_seq(t, "\033(Babc\r\nD"); + assert(tt_cell(t, 0, 0)->text == 'a'); + assert(tt_cell(t, 0, 1)->text == 'D'); + write_seq(t, "\033[H/tmp/proj\r\n$ "); + assert(tt_cell(t, 0, 1)->text == '$'); + assert(tt_cursor_x(t) == 2 && tt_cursor_y(t) == 1); + write_seq(t, "\r>"); + assert(tt_cell(t, 0, 1)->text == '>'); + assert(tt_cursor_x(t) == 1 && tt_cursor_y(t) == 1); + write_seq(t, "\r\n$"); + assert(tt_cell(t, 0, 2)->text == '$'); + assert(tt_cursor_x(t) == 1 && tt_cursor_y(t) == 2); + write_seq(t, "\033[H\033(0lqqk\033(B"); + assert(tt_cell(t, 0, 0)->text == 0x250c); + assert(tt_cell(t, 3, 0)->text == 0x2510); + write_seq(t, "\033[3d\rX\033[4bY"); + assert(tt_cell(t, 4, 2)->text == 'X'); + assert(tt_cell(t, 5, 2)->text == 'Y'); + write_seq(t, "\033[3;3H\033[2X"); + assert(tt_cell(t, 2, 2)->text == ' ' && tt_cell(t, 3, 2)->text == ' '); + assert(tt_cell(t, 4, 2)->text == 'X' && tt_cursor_x(t) == 2); + + write_seq(t, "\033[2J\033[H1234567890"); + assert(tt_cell(t, 9, 0)->text == '0'); + assert(tt_cursor_x(t) == 9 && tt_cursor_y(t) == 0); + write_seq(t, "A"); + assert(tt_cell(t, 0, 1)->text == 'A'); + assert(tt_cursor_x(t) == 1 && tt_cursor_y(t) == 1); + + write_seq(t, "\033[2J\033[Habcdefghij\r\nX"); + assert(tt_cell(t, 9, 0)->text == 'j'); + assert(tt_cell(t, 0, 1)->text == 'X' && tt_cursor_y(t) == 1); + + write_seq(t, "\033[2J\033[H\033[99GZ"); + assert(tt_cell(t, 9, 0)->text == 'Z' && tt_cursor_x(t) == 9); + write_seq(t, "\033[2J\033[9;99HY"); + assert(tt_cursor_y(t) == 3 && tt_cell(t, 9, 3)->text == 'Y'); + + write_seq(t, "\033[2J\033[H\033[?7l1234567890AB"); + assert(tt_cell(t, 9, 0)->text == 'B' && tt_cursor_y(t) == 0); + write_seq(t, "\033[?7h"); + + write_seq(t, "\033[2J\033[Habcd\033[2G\033[4hX\033[4l"); + assert(row_is(t, 0, "aXbcd")); + + write_seq(t, "\033[2;4r\033[?6h\033[2;3H@"); + assert(tt_cell(t, 2, 2)->text == '@'); + write_seq(t, "\033[?6l\033[r"); + + write_seq(t, "\033[2J\033[H\033[3g\033[4G\033H\r\tT"); + assert(tt_cell(t, 3, 0)->text == 'T'); + + write_seq(t, "\033[3;5H\033[5n"); + assert(!strcmp(response, "\033[0n")); + write_seq(t, "\033[6n"); + assert(!strcmp(response, "\033[3;5R")); + + write_seq(t, "\033[2J\033[?69h\033[3;6s\033[?6h\033[Habcdef"); + assert(tt_cell(t, 2, 0)->text == 'a' && tt_cell(t, 5, 0)->text == 'd'); + assert(tt_cell(t, 2, 1)->text == 'e' && tt_cell(t, 3, 1)->text == 'f'); + write_seq(t, "\033[?6;69l"); + + write_seq(t, "\033[?1;5;1004;2026h\033[!p"); + struct term_modes modes = term_modes(t); + assert(!modes.app_cursor && !modes.focus_events); + assert(!modes.synchronized && !modes.reverse_video); + write_seq(t, "kept\033c"); + assert(row_is(t, 0, "")); + + write_seq(t, "\033[2J\033[Hz\033[11b"); + assert(tt_cell(t, 9, 0)->text == 'z' && tt_cell(t, 1, 1)->text == 'z'); + assert(tt_cell(t, 2, 1)->text == ' '); + assert(tt_cursor_x(t) == 2 && tt_cursor_y(t) == 1); + write_seq(t, "\033[2J\033[H1234567890\rA"); + assert(tt_cell(t, 0, 0)->text == 'A'); + write_seq(t, "\033[2;4H\033[2J"); + assert(tt_cursor_x(t) == 3 && tt_cursor_y(t) == 1); + assert(term_modes(t).mouse == TERM_MOUSE_OFF); + write_seq(t, "\033[?1000;1006h"); + assert(term_modes(t).mouse == TERM_MOUSE_NORMAL); + assert(term_modes(t).mouse_encoding == TERM_MOUSE_ENC_SGR); + write_seq(t, "\033[?1002h"); + assert(term_modes(t).mouse == TERM_MOUSE_BUTTON); + assert(term_modes(t).mouse_encoding == TERM_MOUSE_ENC_SGR); + write_seq(t, "\033[?1003h"); + assert(term_modes(t).mouse == TERM_MOUSE_ANY); + write_seq(t, "\033[?1006l"); + assert(term_modes(t).mouse_encoding == TERM_MOUSE_ENC_X10); + write_seq(t, "\033[?1015h"); + assert(term_modes(t).mouse_encoding == TERM_MOUSE_ENC_URXVT); + write_seq(t, "\033[?1016h"); + assert(term_modes(t).mouse_encoding == TERM_MOUSE_ENC_SGR_PIXELS); + write_seq(t, "\033[?1004h"); + term_focus(t, false); + assert(!strcmp(response, "\033[O")); + term_focus(t, true); + assert(!strcmp(response, "\033[I")); + write_seq(t, "\033[?1003l"); + assert(term_modes(t).mouse == TERM_MOUSE_OFF); + response_len = 0; + write_seq(t, "\033]10;?\007"); + assert(!strcmp(response, "\033]10;rgb:ffff/ffff/ffff\033\\")); + write_seq(t, "\033P+q696e646e\033\\"); + assert(!strcmp(response, "\033P0+r696e646e\033\\")); + write_seq(t, "\033P+q6b696e64\033\\"); + assert(!strcmp(response, "\033P1+r6b696e64=1b5b313b3242\033\\")); + term_free(t); + t = tt_new(40, 4); + write_seq(t, "hello 世界 \r\nsecond line\r\nhttps://example.com/path."); + tt_select(t, 9, 0, 0, 0); + char *text = tt_selection_text(t); + assert(text && !strcmp(text, "hello 世界")); + free(text); + tt_select(t, 5, 1, 0, 0); + text = tt_selection_text(t); + assert(text && !strcmp(text, "hello 世界\nsecond")); + free(text); + assert(tt_selected(t, 0, 0) && tt_selected(t, 5, 1)); + assert(!tt_selected(t, 6, 1)); + text = tt_link_at(t, 10, 2); + assert(text && !strcmp(text, "https://example.com/path")); + free(text); + assert(!tt_link_at(t, 10, 1)); + write_seq(t, "\033[?2004h"); + assert(term_modes(t).bracketed_paste); + text = tt_selection_text(t); + assert(text && !strcmp(text, "hello 世界\nsecond")); + free(text); + write_seq(t, "\033[?2004l\r\nlast\r\n"); + assert(!term_modes(t).bracketed_paste); + term_scroll(t, 1); + tt_select(t, 0, 0, 9, 0); + text = tt_selection_text(t); + assert(text && !strcmp(text, "hello 世界")); + free(text); + tt_resize(t, 50, 4); + text = tt_selection_text(t); + assert(text && !strcmp(text, "hello 世界")); + free(text); + term_free(t); + t = tt_new(40, 4); + write_seq(t, "\033[31;44mA\033[>4;0mB\033[>4;1m\033[31mC"); + assert(tt_cell(t, 0, 0)->fg == tt_cell(t, 1, 0)->fg); + assert(tt_cell(t, 0, 0)->bg == tt_cell(t, 1, 0)->bg); + assert(tt_cell(t, 0, 0)->fg == tt_cell(t, 2, 0)->fg); + assert(tt_cell(t, 0, 0)->bg == tt_cell(t, 2, 0)->bg); + assert(tt_cursor_x(t) == 3 && tt_cursor_y(t) == 0); + write_seq(t, "\033[0mD"); + assert(tt_cell(t, 3, 0)->fg == TERM_COLOR_FG); + assert(tt_cell(t, 3, 0)->bg == TERM_COLOR_BG); + term_free(t); + + t = tt_new(10, 4); + write_seq(t, "ABCDEFGHIJ\r\n\r\n\r\n\r\n"); + term_scroll(t, 1); + assert(tt_view_cell(t, 0, 0)->text == 'A'); + assert(tt_view_cell(t, 9, 0)->text == 'J'); + term_scroll_to(t, 0); + + for (int i = 0; i < TEST_SCROLLBACK; i++) + write_seq(t, "xy\r\n"); + term_scroll(t, 1); + assert(tt_view_cell(t, 0, 0)->text == 'x'); + for (int c = 2; c < 10; c++) { + assert(tt_view_cell(t, c, 0)->text == ' '); + assert(tt_view_cell(t, c, 0)->bg == TERM_COLOR_BG); + } + term_free(t); + + t = tt_new(10, 4); + write_seq(t, "\033[41m\033[2K"); + term_color erased_bg = tt_cell(t, 0, 0)->bg; + assert(erased_bg != TERM_COLOR_BG); + assert(tt_cell(t, 9, 0)->bg == erased_bg); + write_seq(t, "\033[m\n\n\n\n"); + term_scroll(t, 1); + assert(tt_view_cell(t, 9, 0)->bg == erased_bg); + assert(tt_view_cell(t, 10, 0)->bg == TERM_COLOR_BG); + term_free(t); + + t = tt_new(10, 4); + write_seq(t, "\033[Hnormal"); + write_seq(t, "\033[?1049h"); + for (int c = 0; c < 10; c++) { + assert(tt_cell(t, c, 0)->text == ' '); + assert(tt_cell(t, c, 0)->bg == TERM_COLOR_BG); + } + write_seq(t, "\033[Halt\n\033[2J"); + write_seq(t, "\033[?1049l"); + assert(tt_cell(t, 0, 0)->text == 'n' && tt_cell(t, 5, 0)->text == 'l'); + term_free(t); + + t = tt_new(20, 8); + write_seq(t, "one\r\ntwo\r\nthree\r\nfour\r\nfive\r\n$ "); + assert(tt_cursor_y(t) == 5 && tt_history(t) == 0); + tt_resize(t, 20, 3); + assert(tt_cursor_y(t) == 2 && tt_history(t) == 3); + assert(row_is(t, 0, "four") && row_is(t, 1, "five") && row_is(t, 2, "$")); + term_scroll(t, 3); + assert(row_is(t, 0, "one") && row_is(t, 1, "two") && row_is(t, 2, "three")); + term_scroll_to(t, 0); + tt_resize(t, 20, 8); + assert(tt_cursor_y(t) == 5 && tt_history(t) == 0); + assert(row_is(t, 0, "one") && row_is(t, 2, "three") && row_is(t, 5, "$")); + assert(row_is(t, 6, "") && row_is(t, 7, "")); + + tt_resize(t, 20, 6); + assert(tt_cursor_y(t) == 5 && tt_history(t) == 0); + assert(row_is(t, 0, "one") && row_is(t, 5, "$")); + term_free(t); + + t = tt_new(20, 6); + write_seq(t, "a\r\nb\r\nc\r\nd\r\ne\r\n"); + write_seq(t, "\033[?1049h\033[6;1Hbottom"); + int before = tt_history(t); + tt_resize(t, 20, 3); + assert(tt_history(t) == before); + tt_resize(t, 20, 6); + assert(tt_history(t) == before); + write_seq(t, "\033[?1049l"); + assert(row_is(t, 0, "a") && row_is(t, 4, "e")); + term_free(t); + + t = tt_new(40, 6); + write_seq(t, "\033[?1049h\033[1;38Hxy"); + tt_resize(t, 12, 6); + write_seq(t, "\033[?1049l"); + assert(tt_cursor_x(t) < tt_cols(t)); + write_seq(t, "Q"); + assert(tt_cell(t, tt_cursor_x(t) - 1, tt_cursor_y(t))->text == 'Q'); + term_free(t); + + t = tt_new(8, 4); + write_seq(t, "abcdefghijkl"); + tt_select(t, 0, 0, 3, 1); + text = tt_selection_text(t); + assert(text && !strcmp(text, "abcdefghijkl")); + free(text); + write_seq(t, "\033[2J\033[Habcd\r\nefgh"); + tt_select(t, 0, 0, 3, 1); + text = tt_selection_text(t); + assert(text && !strcmp(text, "abcd\nefgh")); + free(text); + term_free(t); + + t = tt_new(10, 4); + write_seq(t, "abcdef"); + write_seq(t, "\033[1;3H\033[P"); + assert(row_is(t, 0, "abdef")); + assert(tt_cursor_x(t) == 2 && tt_cursor_y(t) == 0); + write_seq(t, "\033[2P"); + assert(row_is(t, 0, "abf")); + assert(tt_cursor_x(t) == 2 && tt_cursor_y(t) == 0); + write_seq(t, "\033[10P"); + assert(row_is(t, 0, "ab")); + assert(tt_cursor_x(t) == 2 && tt_cursor_y(t) == 0); + term_free(t); + + t = tt_new(8, 4); + write_seq(t, "abcdef"); + write_seq(t, "\033[1;3H\033[@"); + assert(row_is(t, 0, "ab cdef")); + assert(tt_cursor_x(t) == 2 && tt_cursor_y(t) == 0); + write_seq(t, "\033[2@"); + assert(row_is(t, 0, "ab cde")); + term_free(t); + + t = tt_new(10, 4); + assert(term_cursor(t).style == TERM_CURSOR_BLOCK); + assert(term_cursor(t).blink == true); + + write_seq(t, "\033[2 q"); + assert(term_cursor(t).style == TERM_CURSOR_BLOCK); + assert(term_cursor(t).blink == false); + + write_seq(t, "\033[3 q"); + assert(term_cursor(t).style == TERM_CURSOR_UNDERLINE); + assert(term_cursor(t).blink == true); + + write_seq(t, "\033[4 q"); + assert(term_cursor(t).style == TERM_CURSOR_UNDERLINE); + assert(term_cursor(t).blink == false); + + write_seq(t, "\033[5 q"); + assert(term_cursor(t).style == TERM_CURSOR_BAR); + assert(term_cursor(t).blink == true); + + write_seq(t, "\033[6 q"); + assert(term_cursor(t).style == TERM_CURSOR_BAR); + assert(term_cursor(t).blink == false); + + write_seq(t, "\033[0 q"); + assert(term_cursor(t).style == TERM_CURSOR_BLOCK); + assert(term_cursor(t).blink == true); + + write_seq(t, "\033[5 q"); + write_seq(t, "\033[ q"); + assert(term_cursor(t).style == TERM_CURSOR_BLOCK); + assert(term_cursor(t).blink == true); + + write_seq(t, "\033[6 q"); + assert(term_cursor(t).style == TERM_CURSOR_BAR && !term_cursor(t).blink); + write_seq(t, "\033[?1049h"); + write_seq(t, "\033[3 q"); + assert(term_cursor(t).style == TERM_CURSOR_UNDERLINE && term_cursor(t).blink); + write_seq(t, "\033[?1049l"); + assert(term_cursor(t).style == TERM_CURSOR_BAR && !term_cursor(t).blink); + + term_set_cursor_default(t, TERM_CURSOR_UNDERLINE, true); + + write_seq(t, "\033[5 q"); + assert(term_cursor(t).style == TERM_CURSOR_BAR && term_cursor(t).blink); + write_seq(t, "\033[0 q"); + assert(term_cursor(t).style == TERM_CURSOR_UNDERLINE && !term_cursor(t).blink); + + write_seq(t, "\033[2 q"); + assert(term_cursor(t).style == TERM_CURSOR_BLOCK && !term_cursor(t).blink); + write_seq(t, "\033[ q"); + assert(term_cursor(t).style == TERM_CURSOR_UNDERLINE && !term_cursor(t).blink); + + write_seq(t, "\033[1 q"); + assert(term_cursor(t).style == TERM_CURSOR_BLOCK && term_cursor(t).blink); + write_seq(t, "\033[!p"); + assert(term_cursor(t).style == TERM_CURSOR_UNDERLINE && !term_cursor(t).blink); + term_free(t); + + t = tt_new(20, 4); + write_seq(t, "\033[1;2;3;4;9;53mA\033[21mB\033[4:2mC\033[22;24mD\033[0mE"); + const uint16_t styles = TERM_ATTR_BOLD | TERM_ATTR_FAINT | TERM_ATTR_ITALIC | + TERM_ATTR_STRIKE | TERM_ATTR_OVERLINE; + assert((tt_cell(t, 0, 0)->attrs & styles) == styles); + assert(term_attr_underline(tt_cell(t, 0, 0)->attrs) == TERM_UNDERLINE_SINGLE); + assert(term_attr_underline(tt_cell(t, 1, 0)->attrs) == TERM_UNDERLINE_DOUBLE); + assert(term_attr_underline(tt_cell(t, 2, 0)->attrs) == TERM_UNDERLINE_DOUBLE); + assert(!(tt_cell(t, 3, 0)->attrs & (TERM_ATTR_BOLD | TERM_ATTR_FAINT))); + assert(term_attr_underline(tt_cell(t, 3, 0)->attrs) == TERM_UNDERLINE_NONE); + assert(tt_cell(t, 4, 0)->attrs == 0); + term_free(t); + + t = tt_new(20, 4); + write_seq(t, "\033[30mA\033[0;1;30mB\033[0;90mC"); + assert(tt_cell(t, 0, 0)->fg == tt_cell(t, 1, 0)->fg); + assert(tt_cell(t, 1, 0)->fg != tt_cell(t, 2, 0)->fg); + assert(!(tt_cell(t, 0, 0)->attrs & TERM_ATTR_BOLD)); + assert(tt_cell(t, 1, 0)->attrs & TERM_ATTR_BOLD); + term_free(t); + + const char *colon_rgb = "\033[48;2;145;2;34m\033[38;2;12;34;56mA" + "\033[38:2::12:34:56mB\033[;mC"; + for (size_t split = 0; split <= strlen(colon_rgb); split++) { + t = tt_new(20, 4); + tt_write(t, colon_rgb, split); + tt_write(t, colon_rgb + split, strlen(colon_rgb) - split); + assert(row_is(t, 0, "ABC")); + for (int x = 0; x < 2; x++) { + assert(tt_cell(t, x, 0)->fg == 0xFF0C2238); + assert(tt_cell(t, x, 0)->bg == 0xFF910222); + } + assert(tt_cell(t, 2, 0)->fg == TERM_COLOR_FG); + assert(tt_cell(t, 2, 0)->bg == TERM_COLOR_BG); + term_free(t); + } + + t = tt_new(20, 4); + write_seq(t, "\033[38;2;246;226;183;49mX"); + assert(tt_cell(t, 0, 0)->fg == 0xFFF6E2B7); + assert(tt_cell(t, 0, 0)->bg == TERM_COLOR_BG); + term_free(t); + + t = tt_new(20, 4); + const char *clusters = "(\xc2\xa9\xef\xb8\x8f)\xe2\x98\xba\xef\xb8\x8e" + "\xf0\x9f\x91\xa8\xe2\x80\x8d\xf0\x9f\x8c\xbe"; + write_seq(t, clusters); + + assert(tt_cursor_x(t) == 7); + assert((tt_cell(t, 1, 0)->attrs & TERM_ATTR_EMOJI) && tt_cell(t, 1, 0)->width == 2); + assert(tt_cell(t, 2, 0)->width == 0); + assert((tt_cell(t, 4, 0)->attrs & TERM_ATTR_TEXT) && tt_cell(t, 4, 0)->width == 1); + assert((tt_cell(t, 5, 0)->attrs & TERM_ATTR_EMOJI) && tt_cell(t, 5, 0)->width == 2); + assert(tt_cell(t, 6, 0)->width == 0); + + tt_select(t, 0, 0, 6, 0); + text = tt_selection_text(t); + assert(text && !strcmp(text, clusters)); + free(text); + term_free(t); + + /* BS with a pending wrap clears the wrap and moves left from the last + * column, as in xterm and the VT100. */ + t = tt_new(5, 2); + write_seq(t, "12345"); + assert(row_is(t, 0, "12345")); + assert(tt_cursor_x(t) == 4 && tt_cursor_y(t) == 0); + write_seq(t, "\b"); + assert(tt_cursor_x(t) == 3 && tt_cursor_y(t) == 0); + write_seq(t, "X"); + assert(row_is(t, 0, "123X5")); + assert(tt_cursor_x(t) == 4 && tt_cursor_y(t) == 0); + write_seq(t, "\b \b"); + assert(row_is(t, 0, "123 5")); + assert(tt_cursor_x(t) == 3 && tt_cursor_y(t) == 0); + term_free(t); + + /* A renderer sees each cursor move within the row. */ + t = tt_new(10, 2); + struct term_snapshot *snap = term_snapshot_new(); + write_seq(t, "abc"); + uint64_t version = view_version(snap, t); + write_seq(t, "\b"); + assert(tt_cursor_x(t) == 2 && view_version(snap, t) != version); + version = view_version(snap, t); + write_seq(t, "\r"); + assert(tt_cursor_x(t) == 0 && view_version(snap, t) != version); + version = view_version(snap, t); + write_seq(t, "\t"); + assert(tt_cursor_x(t) == 8 && view_version(snap, t) != version); + term_snapshot_free(snap); + term_free(t); + + t = tt_new(20, 2); + tt_write(t, "hello world", 11); + assert(row_is(t, 0, "hello world")); + assert(tt_cursor_x(t) == 11 && tt_cursor_y(t) == 0); + term_free(t); + + title[0] = 0; + t = tt_new_opts(10, 4, &title_cb); + write_seq(t, "\033]0;My Shell\007"); + assert(!strcmp(title, "My Shell")); + write_seq(t, "\033]2;Second Title\033\\"); + assert(!strcmp(title, "Second Title")); + write_seq(t, "\033]1;Icon Name\007"); + assert(!strcmp(title, "Icon Name")); + term_free(t); + + t = tt_new(20, 4); + write_seq(t, "$ /mnt/work"); + const char *round_trip = "\033[?1049h\033[?1049l"; + tt_write(t, round_trip, strlen(round_trip)); + assert(row_is(t, 0, "$ /mnt/work")); + term_free(t); + + /* A renderer sees both switches between the screens. */ + t = tt_new(20, 4); + snap = term_snapshot_new(); + write_seq(t, "$ /mnt/work"); + version = view_version(snap, t); + write_seq(t, "\033[?1049h"); + assert(view_version(snap, t) != version); + version = view_version(snap, t); + write_seq(t, "\033[?1049l"); + assert(view_version(snap, t) != version); + assert(row_is(t, 0, "$ /mnt/work")); + term_snapshot_free(snap); + term_free(t); + + t = tt_new(20, 4); + write_seq(t, "output"); + write_seq(t, "\r\033[K../"); + assert(row_is(t, 0, "../")); + term_free(t); + + t = tt_new(80, 24); + assert(!term_modes(t).alternate); + write_seq(t, "\033[?1049h"); + assert(term_modes(t).alternate); + write_seq(t, "\033[?1049l"); + assert(!term_modes(t).alternate); + term_free(t); + + const char *color_runs[] = { + "\033[1;1H\033[48;2;10;20;30m\033[38;2;200;210;220ma" + "\033[48;2;100;110;120m\033[38;2;230;240;250mb", + "\033[1;1H\033[38;2;200;210;220m\033[48;2;10;20;30ma" + "\033[38;2;230;240;250m\033[48;2;100;110;120mb", + }; + for (size_t run = 0; run < sizeof(color_runs) / sizeof(color_runs[0]); run++) { + const char *seq = color_runs[run]; + size_t len = strlen(seq); + for (size_t split = 0; split <= len; split++) { + t = tt_new(100, 2); + tt_write(t, seq, split); + tt_write(t, seq + split, len - split); + assert(row_is(t, 0, "ab")); + assert(tt_cursor_x(t) == 2 && tt_cursor_y(t) == 0); + assert(tt_cell(t, 0, 0)->fg == 0xffc8d2dc); + assert(tt_cell(t, 0, 0)->bg == 0xff0a141e); + assert(tt_cell(t, 1, 0)->fg == 0xffe6f0fa); + assert(tt_cell(t, 1, 0)->bg == 0xff646e78); + for (int col = 2; col < 100; col++) + assert(tt_cell(t, col, 0)->bg == TERM_COLOR_BG); + assert(row_is(t, 1, "")); + term_free(t); + } + } + + puts("PASS: terminal regression tests"); + return 0; +} diff --git a/tests/unicode/unicode_test.c b/tests/unicode/unicode_test.c new file mode 100644 index 0000000..7fc6e3e --- /dev/null +++ b/tests/unicode/unicode_test.c @@ -0,0 +1,138 @@ +#include "term/unicode/utf8.h" +#include <assert.h> +#include <stdio.h> +#include <string.h> + +static void test_utf8_decode_one(void) +{ + static const struct { const char *bytes; int len; uint32_t cp; } valid[] = { + {"\0", 1, 0}, {"\x7f", 1, 0x7f}, + {"\xc2\x80", 2, 0x80}, {"\xdf\xbf", 2, 0x7ff}, + {"\xe0\xa0\x80", 3, 0x800}, {"\xed\x9f\xbf", 3, 0xd7ff}, + {"\xee\x80\x80", 3, 0xe000}, {"\xef\xbf\xbf", 3, 0xffff}, + {"\xf0\x90\x80\x80", 4, 0x10000}, + {"\xf4\x8f\xbf\xbf", 4, 0x10ffff}, + }; + for (size_t i = 0; i < sizeof(valid) / sizeof(*valid); i++) { + uint32_t cp = 0xdeadbeef; + for (int n = 0; n < valid[i].len; n++) { + int r = utf8_decode_one(valid[i].bytes, n, &cp); + assert(r == 0); + assert(cp == 0xdeadbeef); + } + int r = utf8_decode_one(valid[i].bytes, valid[i].len, &cp); + assert(r == valid[i].len); + assert(cp == valid[i].cp); + char followed[8]; + memcpy(followed, valid[i].bytes, valid[i].len); + memcpy(followed + valid[i].len, "next", 4); + r = utf8_decode_one(followed, valid[i].len + 4, &cp); + assert(r == valid[i].len); + assert(cp == valid[i].cp); + } + static const char *const invalid[] = { + "\x80", "\xbf", "\xc0\x80", "\xc1\xbf", "\xc2X", + "\xe0\x9f\xbf", "\xed\xa0\x80", "\xed\xbf\xbf", "\xe1\x80X", + "\xf0\x8f\xbf\xbf", "\xf4\x90\x80\x80", "\xf5\x80\x80\x80", + "\xf8\x88\x80\x80\x80", "\xff", "\xf1\x80\x80X", + }; + for (size_t i = 0; i < sizeof(invalid) / sizeof(*invalid); i++) { + uint32_t cp = 0xdeadbeef; + int r = utf8_decode_one(invalid[i], strlen(invalid[i]), &cp); + assert(r == 0); + assert(cp == 0xdeadbeef); + } +} + +static void test_scalar_roundtrips(void) +{ + for (uint32_t cp = 0; cp <= 0x10ffff; cp++) { + if (cp >= 0xd800 && cp <= 0xdfff) continue; + char bytes[UTF8_MAX_BYTES]; + int len = utf8_encode(cp, bytes); + uint32_t scalar = 0, streamed = 0, state = UTF8_ACCEPT; + int r = utf8_decode_one(bytes, (size_t)len, &scalar); + assert(r == len); + assert(scalar == cp); + for (int i = 0; i < len; i++) { + utf8_decode(&state, &streamed, (unsigned char)bytes[i]); + if (i + 1 < len) assert(state != UTF8_ACCEPT && state != UTF8_REJECT); + } + assert(state == UTF8_ACCEPT && streamed == cp); + } + static const uint32_t invalid[] = {0xd800, 0xdfff, 0x110000, UINT32_MAX}; + for (size_t i = 0; i < sizeof(invalid) / sizeof(*invalid); i++) { + char bytes[UTF8_MAX_BYTES]; + int len = utf8_encode(invalid[i], bytes); + uint32_t cp = 0; + int r = utf8_decode_one(bytes, len, &cp); + assert(len == 3 && r == len && cp == 0xfffd); + } +} + +static void test_utf8_sanitize(void) +{ + char dst[64]; + + utf8_sanitize(dst, sizeof(dst), "hello \xe2\x82\xac\xf0\x9f\x98\x80"); + assert(!strcmp(dst, "hello \xe2\x82\xac\xf0\x9f\x98\x80")); + + utf8_sanitize(dst, sizeof(dst), "a\xff\x80" "b\xed\xa0\x80" "c\xe2\x82"); + assert(!strcmp(dst, "abc")); + + utf8_sanitize(dst, sizeof(dst), ""); + assert(!strcmp(dst, "")); + utf8_sanitize(dst, sizeof(dst), "\xff\xfe"); + assert(!strcmp(dst, "")); + + utf8_sanitize(dst, 6, "ab\xe2\x82\xac" "cd"); + assert(!strcmp(dst, "ab\xe2\x82\xac")); + utf8_sanitize(dst, 5, "ab\xe2\x82\xac" "cd"); + assert(!strcmp(dst, "abcd")); + memset(dst, 'x', sizeof(dst)); + utf8_sanitize(dst, 4, "abcdef"); + assert(!strcmp(dst, "abc")); + memset(dst, 'x', sizeof(dst)); + utf8_sanitize(dst, 0, "abc"); + assert(dst[0] == 'x'); +} + +/* Each maximal invalid subpart gives U+FFFD. The byte that ends a subpart + * stays unused. */ +static void test_ground_invalid_subparts(void) +{ + uint32_t state = UTF8_ACCEPT, codep = 0, cp = 0xdead; + int n = utf8_decode_ground(&state, &codep, "\xe2\x80\x1b\xac", 4, &cp); + assert(n == 2); + assert(cp == 0xFFFD); + assert(state == UTF8_ACCEPT); + n = utf8_decode_ground(&state, &codep, "\xac", 1, &cp); + assert(n == 1); + assert(cp == 0xFFFD); + assert(state == UTF8_ACCEPT); + n = utf8_decode_ground(&state, &codep, "\xe2\x82z", 3, &cp); + assert(n == 2); + assert(cp == 0xFFFD); + assert(state == UTF8_ACCEPT); + n = utf8_decode_ground(&state, &codep, "\xe2\x82", 2, &cp); + assert(n == 2); + assert(cp == 0); + assert(state != UTF8_ACCEPT); + + state = UTF8_ACCEPT; + codep = 0; + n = utf8_decode_ground(&state, &codep, "\xe2\x82\xac", 3, &cp); + assert(n == 3); + assert(cp == 0x20ac); + assert(state == UTF8_ACCEPT); +} + +int main(void) +{ + test_utf8_decode_one(); + test_scalar_roundtrips(); + test_utf8_sanitize(); + test_ground_invalid_subparts(); + puts("PASS: Unicode codec tests"); + return 0; +} diff --git a/tests/view_test.c b/tests/view_test.c new file mode 100644 index 0000000..5e17450 --- /dev/null +++ b/tests/view_test.c @@ -0,0 +1,475 @@ +#include <assert.h> +#include "term_api.h" + +#include <limits.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> + +/* + * The scrollback view: the viewport can leave the live screen, and the + * terminal then shows history rows. These tests cover what the view does + * while the terminal keeps working underneath it. + */ + +/* The tests use small terminals, so a row fits in a short buffer. */ +#define VIEW_TEXT_CAP 256 + +/* The text of a view row, trailing blanks cut. */ +static void view_row_text(struct term *t, int row, char *buf, size_t cap) +{ + const struct term_cell *cells = term_snapshot_row(tt_snapshot_for(t), row); + size_t n = 0; + for (int c = 0; c < tt_cols(t) && n + 1 < cap; c++) { + uint32_t cp = tt_cell_cp(t, &cells[c]); + buf[n++] = !cp ? ' ' : cp < 0x80 ? (char)cp : '?'; + } + while (n > 0 && buf[n - 1] == ' ') + n--; + buf[n] = 0; +} + +static bool view_row_is(struct term *t, int row, const char *want) +{ + char got[VIEW_TEXT_CAP]; + view_row_text(t, row, got, sizeof(got)); + return !strcmp(got, want); +} + +/* Every view row holds the text of the same row in want. A view row that + * resolves to no row at all fails the text compare. */ +static void assert_view_rows_equal(struct term *t, struct term *want) +{ + for (int row = 0; row < tt_rows(t); row++) { + char got[VIEW_TEXT_CAP], expect[VIEW_TEXT_CAP]; + view_row_text(t, row, got, sizeof(got)); + view_row_text(want, row, expect, sizeof(expect)); + assert(!strcmp(got, expect)); + } +} + +/* The view offset, in range, and the same as the snapshot reports. The two + * must come from one field: a second copy of the offset is what let the + * scrollbar and the rows disagree. */ +static void assert_offset_agrees(struct term *t) +{ + const struct term_view *v = term_snapshot_view(tt_snapshot_for(t)); + struct term_scrollback sb = term_scrollback(t); + assert(sb.offset == v->scroll_offset); + assert(sb.offset >= 0); + assert(sb.offset <= sb.total); +} + +/* A terminal whose history is small enough to fill. */ +static struct term *small_history_term(int cols, int rows, int scrollback) +{ + struct term_options opt = { + .size = { .x = cols, .y = rows }, + .scrollback = scrollback, + }; + struct term *t = term_new(&opt); + assert(t); + return t; +} + +/* Fill the history with count lines of "Lnn" and leave the cursor on the + * last screen row. */ +static void feed_lines(struct term *t, const char *tag, int count) +{ + char line[64]; + for (int i = 0; i < count; i++) { + int len = snprintf(line, sizeof(line), "%s%02d\r\n", tag, i); + tt_write(t, line, (size_t)len); + } +} + +/* Output that arrives while the view sits in the history leaves the view + * on the same text. */ +static void view_holds_text_while_output_arrives(void) +{ + struct term *t = tt_new(12, 5); + struct term *reference = tt_new(12, 5); + feed_lines(t, "L", 15); + feed_lines(reference, "L", 15); + term_scroll(t, 2); + term_scroll(reference, 2); + assert(tt_view_offset(t) == 2); + assert_view_rows_equal(t, reference); + + feed_lines(t, "M", 4); + feed_lines(reference, "M", 4); + assert_offset_agrees(t); + /* Nothing was evicted, so the view still shows the rows it showed. */ + assert_view_rows_equal(t, reference); + term_free(t); + term_free(reference); +} + +/* A view that sits at the top of the history keeps the oldest text there + * when more output arrives. The history grows by a row, so the offset + * grows with it. */ +static void view_at_the_top_keeps_the_oldest_text(void) +{ + const int scrollback = 20; + struct term *t = small_history_term(12, 5, scrollback); + struct term *reference = small_history_term(12, 5, scrollback); + feed_lines(t, "L", 12); + feed_lines(reference, "L", 12); + term_scroll_to(t, INT_MAX); + term_scroll_to(reference, INT_MAX); + int total = tt_history(t); + assert(total > 0 && total + tt_rows(t) <= scrollback); + assert(tt_view_offset(t) == total); + assert(view_row_is(t, 0, "L00")); + + /* Three more lines with room in the history: the oldest line stays on + * the top row of the view, and the offset grows with the history. */ + feed_lines(t, "M", 3); + feed_lines(reference, "M", 3); + assert_offset_agrees(t); + assert(tt_history(t) == total + 3); + assert(tt_view_offset(t) == total + 3); + assert(view_row_is(t, 0, "L00")); + assert_view_rows_equal(t, reference); + term_free(t); + term_free(reference); +} + +/* Output larger than the history evicts the rows the view was anchored to. + * The view then shows the oldest text the terminal still keeps, and the + * offset stays inside the history. */ +static void view_survives_output_larger_than_history(void) +{ + const int scrollback = 20; + struct term *t = small_history_term(12, 5, scrollback); + struct term *reference = small_history_term(12, 5, scrollback); + feed_lines(t, "L", 15); + feed_lines(reference, "L", 15); + term_scroll(t, 3); + + feed_lines(t, "M", 40); + feed_lines(reference, "M", 40); + assert_offset_agrees(t); + assert(tt_history(t) == tt_history(reference)); + assert(tt_history(t) == scrollback); + assert(tt_view_offset(t) > 0); + + /* The top of the history is where the view ends up. */ + term_scroll(reference, INT_MAX); + assert(tt_view_offset(reference) == tt_history(reference)); + assert(tt_view_offset(t) == tt_history(t)); + assert_view_rows_equal(t, reference); + term_free(t); + term_free(reference); +} + +/* An eviction can drop a line that wraps to more rows than the push adds. + * The offset then stops at the oldest row the terminal still keeps, and + * never sits above the history. */ +static void view_survives_eviction_of_a_wrapped_line(void) +{ + const int scrollback = 20; + struct term *t = small_history_term(12, 5, scrollback); + struct term *reference = small_history_term(12, 5, scrollback); + /* The oldest line wraps to three rows. */ + const char *wrapped = "0123456789012345678901234\r\n"; + tt_puts(t, wrapped); + tt_puts(reference, wrapped); + feed_lines(t, "s", 22); + feed_lines(reference, "s", 22); + term_scroll_to(t, INT_MAX); + term_scroll_to(reference, INT_MAX); + assert_offset_agrees(t); + + /* The next line evicts the wrapped one: three rows go, one arrives. */ + for (int i = 0; i < 4; i++) { + tt_puts(t, "n\r\n"); + tt_puts(reference, "n\r\n"); + assert_offset_agrees(t); + assert(tt_view_offset(t) <= tt_history(t)); + } + /* The view sits on the oldest text the terminal still keeps. */ + term_scroll_to(reference, INT_MAX); + assert(tt_view_offset(t) == tt_view_offset(reference)); + assert_view_rows_equal(t, reference); + term_free(t); + term_free(reference); +} + +/* A resize keeps the same retained text at the top of the view. */ +static void view_keeps_anchor_on_resize(void) +{ + static const struct { + int cols, rows; + } sizes[] = { { 12, 6 }, { 20, 5 }, { 6, 5 } }; + for (size_t i = 0; i < sizeof(sizes) / sizeof(sizes[0]); i++) { + struct term *t = tt_new(12, 5); + struct term *reference = tt_new(12, 5); + feed_lines(t, "L", 15); + feed_lines(reference, "L", 15); + term_scroll(t, 3); + assert(tt_view_offset(t) == 3); + assert(tt_resize(t, sizes[i].cols, sizes[i].rows)); + assert(tt_resize(reference, sizes[i].cols, sizes[i].rows)); + assert_offset_agrees(t); + assert(tt_view_offset(t) > 0); + assert(view_row_is(t, 0, "L08")); + term_scroll_to(t, 0); + assert_view_rows_equal(t, reference); + term_free(t); + term_free(reference); + } +} + +/* The alternate screen has no history to show. The normal screen comes + * back at its own bottom. */ +static void view_follows_the_alternate_screen(void) +{ + struct term *t = tt_new(12, 5); + feed_lines(t, "L", 15); + term_scroll(t, 3); + assert(tt_view_offset(t) == 3); + + tt_puts(t, "\033[?1049hALT"); + assert_offset_agrees(t); + assert(tt_view_offset(t) == 0); + assert(view_row_is(t, 0, "ALT")); + + tt_puts(t, "\033[?1049l"); + assert_offset_agrees(t); + assert(tt_view_offset(t) == 0); + assert(view_row_is(t, 0, "L11")); + term_free(t); +} + +/* The view offset has one owner. Every operation that moves or clears the + * view leaves the public offset and the snapshot offset equal. */ +static void view_offset_has_one_owner(void) +{ + struct term *t = tt_new(12, 5); + feed_lines(t, "L", 15); + assert_offset_agrees(t); + term_scroll(t, 3); + assert_offset_agrees(t); + term_scroll_to(t, INT_MAX); + assert_offset_agrees(t); + assert(tt_view_offset(t) == tt_history(t)); + term_scroll(t, INT_MIN); + assert_offset_agrees(t); + assert(tt_view_offset(t) == 0); + term_free(t); +} + +/* ED2 and ED3 return the view to the live screen. RIS keeps the offset, as + * it did before the view had an anchor of its own. */ +static void view_returns_to_bottom_on_erase(void) +{ + struct term *t = tt_new(12, 5); + feed_lines(t, "L", 15); + term_scroll(t, 3); + assert(tt_view_offset(t) == 3); + + tt_puts(t, "\033[2J"); + assert_offset_agrees(t); + assert(tt_view_offset(t) == 0); + + term_scroll(t, 2); + assert(tt_view_offset(t) == 2); + tt_puts(t, "\033[3J"); + assert_offset_agrees(t); + assert(tt_view_offset(t) == 0); + assert(tt_history(t) == 0); + term_free(t); + + t = tt_new(12, 5); + feed_lines(t, "L", 15); + term_scroll(t, 3); + int before = tt_view_offset(t); + tt_puts(t, "\033c"); + assert_offset_agrees(t); + assert(tt_view_offset(t) == before); + term_free(t); +} + +/* Rewrap keeps the cell that selected the anchored segment in view. */ +static void wrapped_anchor_survives_resize(void) +{ + struct term *t = tt_new(10, 3); + tt_puts(t, "abcdefghijKLMNOPQRSTuvwxyz\r\n"); + feed_lines(t, "L", 10); + term_scroll_to(t, INT_MAX); + term_scroll(t, -1); + assert(view_row_is(t, 0, "KLMNOPQRST")); + tt_select(t, 0, 0, 2, 0); + char *selected = tt_selection_text(t); + assert(selected && !strcmp(selected, "KLM")); + free(selected); + assert(tt_resize(t, 6, 3)); + assert_offset_agrees(t); + assert(view_row_is(t, 0, "ghijKL")); + selected = tt_selection_text(t); + assert(selected && !strcmp(selected, "KLM")); + free(selected); + feed_lines(t, "N", 5); + assert(view_row_is(t, 0, "ghijKL")); + assert(tt_resize(t, 13, 3)); + assert(view_row_is(t, 0, "abcdefghijKLM")); + assert(tt_resize(t, 6, 3)); + assert(view_row_is(t, 0, "ghijKL")); + assert_offset_agrees(t); + term_free(t); +} + +/* A wide glyph leaves a wrap pad. That pad does not move the anchor. */ +static void wide_anchor_survives_resize(void) +{ + struct term *t = tt_new(5, 3); + tt_puts(t, "abcd日本語XYZ\r\n"); + feed_lines(t, "L", 10); + term_scroll_to(t, INT_MAX); + term_scroll(t, -1); + assert(tt_cell_cp(t, tt_view_cell(t, 0, 0)) == 0x65e5); + assert(tt_resize(t, 3, 3)); + assert_offset_agrees(t); + assert(tt_cell_cp(t, tt_view_cell(t, 0, 0)) == 'd'); + assert(tt_cell_cp(t, tt_view_cell(t, 1, 0)) == 0x65e5); + assert(tt_resize(t, 8, 3)); + assert(tt_cell_cp(t, tt_view_cell(t, 0, 0)) == 'a'); + assert(tt_resize(t, 3, 3)); + assert(tt_cell_cp(t, tt_view_cell(t, 1, 0)) == 0x65e5); + term_free(t); +} + +/* Incremental and new snapshots must show the same retained rows. */ +static void snapshots_agree_after_eviction_and_rewrap(void) +{ + struct term *t = small_history_term(11, 5, 23); + struct term_snapshot *persistent = term_snapshot_new(); + assert(persistent); + for (int n = 0; n < 120; n++) { + char line[120]; + snprintf(line, sizeof(line), "%03d abcdefghijklmnop 日本語 café\r\n", n); + tt_puts(t, line); + if (n % 13 == 0) + assert(tt_resize(t, 6 + n % 17, 5)); + int total = tt_history(t); + term_scroll_to(t, n % (total + 1)); + assert(term_snapshot_update(persistent, t)); + struct term_snapshot *fresh = term_snapshot_new(); + assert(fresh && term_snapshot_update(fresh, t)); + struct term_vector2i size = term_size(t); + for (int y = 0; y < size.y; y++) { + const struct term_cell *a = term_snapshot_row(persistent, y); + const struct term_cell *b = term_snapshot_row(fresh, y); + for (int x = 0; x < size.x; x++) { + assert(a[x].fg == b[x].fg && a[x].bg == b[x].bg); + assert(a[x].width == b[x].width && a[x].attrs == b[x].attrs); + uint32_t acp[32], bcp[32]; + int an = term_snapshot_cluster(persistent, a[x].text, acp, 32); + int bn = term_snapshot_cluster(fresh, b[x].text, bcp, 32); + assert(an == bn); + if (an) + assert(!memcmp(acp, bcp, (size_t)an * sizeof(*acp))); + else + assert(a[x].text == b[x].text); + } + assert(term_snapshot_row_key(persistent, y) == term_snapshot_row_key(fresh, y)); + } + assert_offset_agrees(t); + term_snapshot_free(fresh); + } + term_snapshot_free(persistent); + term_free(t); +} + +static void snapshots_agree_after_prefix_eviction(void) +{ + struct term *t = tt_new(1000, 3); + char line[1000]; + for (int i = 0; i < 9500; i++) { + memset(line, 'A' + i % 26, sizeof(line)); + tt_write(t, line, sizeof(line)); + } + tt_write(t, "\r\n", 2); + term_scroll(t, INT_MAX); + struct term_snapshot *persistent = term_snapshot_new(); + assert(persistent && term_snapshot_update(persistent, t)); + uint64_t old_key = term_snapshot_row_key(persistent, 0); + memset(line, 'z', sizeof(line)); + for (int i = 0; i < 1200; i++) { + tt_write(t, line, sizeof(line)); + tt_write(t, "\r\n", 2); + } + assert(term_snapshot_update(persistent, t)); + assert(term_snapshot_row_key(persistent, 0) != old_key); + struct term_snapshot *fresh = term_snapshot_new(); + assert(fresh && term_snapshot_update(fresh, t)); + for (int y = 0; y < 3; y++) { + const struct term_cell *a = term_snapshot_row(persistent, y); + const struct term_cell *b = term_snapshot_row(fresh, y); + for (int x = 0; x < 1000; x++) + assert(a[x].text == b[x].text && a[x].fg == b[x].fg && a[x].bg == b[x].bg); + assert(term_snapshot_row_key(persistent, y) == term_snapshot_row_key(fresh, y)); + } + term_snapshot_free(fresh); + term_snapshot_free(persistent); + term_free(t); +} + +/* A line that needs more than one storage part stays scannable. Every + * scroll offset names one history row, and every view row shows the text + * of the history row the offset puts there. */ +static void view_scrolls_a_line_that_needs_several_parts(void) +{ + const int cols = 40, rows = 6, cells = 2000000; + struct term_options opt = { .size = { .x = cols, .y = rows }, .scrollback = 60000 }; + struct term *t = term_new(&opt); + assert(t); + char *big = malloc((size_t)cells); + assert(big); + for (int i = 0; i < cells; i++) + big[i] = (char)('0' + i % 10); + tt_write(t, big, (size_t)cells); + tt_write(t, "\r\n", 2); + free(big); + + /* The rows still on the screen are not history rows. */ + int total = tt_history(t); + assert(total == cells / cols - (rows - 1)); + struct term_snapshot *snap = term_snapshot_new(); + assert(snap); + for (int off = 1; off <= total; off += 5) { + term_scroll_to(t, off); + assert(tt_view_offset(t) == off); + assert(term_snapshot_update(snap, t)); + for (int y = 0; y < rows; y++) { + int row = total - off + y; + if (row >= total) + continue; + const struct term_cell *line = term_snapshot_row(snap, y); + for (int x = 0; x < cols; x++) + assert(line[x].text == (uint32_t)('0' + (row * cols + x) % 10)); + } + } + term_snapshot_free(snap); + term_free(t); +} + +int main(void) +{ + snapshots_agree_after_prefix_eviction(); + wrapped_anchor_survives_resize(); + wide_anchor_survives_resize(); + snapshots_agree_after_eviction_and_rewrap(); + view_holds_text_while_output_arrives(); + view_at_the_top_keeps_the_oldest_text(); + view_survives_output_larger_than_history(); + view_survives_eviction_of_a_wrapped_line(); + view_keeps_anchor_on_resize(); + view_follows_the_alternate_screen(); + view_offset_has_one_owner(); + view_scrolls_a_line_that_needs_several_parts(); + view_returns_to_bottom_on_erase(); + printf("PASS: scrollback view tests\n"); + return 0; +} diff --git a/tests/write_fuzz_test.c b/tests/write_fuzz_test.c new file mode 100644 index 0000000..8c1475a --- /dev/null +++ b/tests/write_fuzz_test.c @@ -0,0 +1,119 @@ +/* + * Differential fuzz test for term_feed. Each seed makes random terminal + * output: text, controls, CSI and ESC sequences, raw bytes, and resizes. + * One terminal gets the output in a few large writes; the bulk fast paths + * handle it. A second terminal gets it one byte at a time; the generic + * parser handles it. The two must show the same cursor, screen, and + * scrollback. + */ +#include "term_api.h" +#include "fuzz_gen.h" + +#include <assert.h> +#include <stdint.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> + +/* Each terminal has its own snapshot, so cells of both stay readable at + * the same time. */ +static struct term_snapshot *snap_bulk, *snap_bytes; + +/* The code points of a cell's text. A cluster has handles that differ + * between snapshots, so the test compares its code points. */ +static int cell_text(const struct term_snapshot *s, const struct term_cell *cell, + uint32_t *cps, int cap) +{ + int n = term_snapshot_cluster(s, cell->text, cps, cap); + return n < cap ? n : cap; +} + +static bool same_cell(const struct term_cell *a, const struct term_cell *b) +{ + if (a->fg != b->fg || a->bg != b->bg || a->attrs != b->attrs || a->width != b->width) + return false; + uint32_t ta[16], tb[16]; + int na = cell_text(snap_bulk, a, ta, 16); + int nb = cell_text(snap_bytes, b, tb, 16); + return na == nb && memcmp(ta, tb, (size_t)na * sizeof(ta[0])) == 0; +} + +static void assert_same(struct term *bulk, struct term *bytes, int seed, int step) +{ + int cols = tt_cols(bulk), rows = tt_rows(bulk); + int history = tt_history(bulk); + bool same = tt_cursor_x(bulk) == tt_cursor_x(bytes) && + tt_cursor_y(bulk) == tt_cursor_y(bytes) && + history == tt_history(bytes); + term_scroll(bulk, history); + term_scroll(bytes, history); + for (int page = 0; same && page <= history; page += rows) { + bool copied = term_snapshot_update(snap_bulk, bulk); + copied &= term_snapshot_update(snap_bytes, bytes); + assert(copied); + for (int y = 0; same && y < rows; y++) { + const struct term_cell *ra = term_snapshot_row(snap_bulk, y); + const struct term_cell *rb = term_snapshot_row(snap_bytes, y); + for (int x = 0; x < cols; x++) { + const struct term_cell *a = &ra[x], *b = &rb[x]; + if (!same_cell(a, b)) { + fprintf(stderr, "seed %d step %d: page %d row %d col %d: " + "bulk %x/%x/%x/%x/%u, bytes %x/%x/%x/%x/%u\n", + seed, step, page, y, x, a->text, a->fg, a->bg, a->attrs, + a->width, b->text, b->fg, b->bg, b->attrs, b->width); + same = false; + break; + } + } + } + term_scroll(bulk, -rows); + term_scroll(bytes, -rows); + } + term_scroll_to(bulk, 0); + term_scroll_to(bytes, 0); + if (!same) + fprintf(stderr, "seed %d step %d: cursor %d,%d vs %d,%d, history %d vs %d\n", + seed, step, tt_cursor_x(bulk), tt_cursor_y(bulk), + tt_cursor_x(bytes), tt_cursor_y(bytes), + history, tt_history(bytes)); + assert(same); +} + +int main(int argc, char **argv) +{ + int seeds = argc > 1 ? atoi(argv[1]) : 50000; + for (int seed = 1; seed <= seeds; seed++) { + rng = 0x9e3779b97f4a7c15ull * (uint64_t)seed; + int cols = 4 + rn(30), rows = 2 + rn(10); + struct term *bulk = tt_new(cols, rows); + struct term *bytes = tt_new(cols, rows); + /* A new terminal can get the address of a freed one, so each seed + * uses new snapshots. */ + snap_bulk = term_snapshot_new(); + snap_bytes = term_snapshot_new(); + assert(bulk && bytes && snap_bulk && snap_bytes); + for (int step = 0; step < 6; step++) { + blen = 0; + gen(20 + rn(60), cols, rows); + size_t split = blen ? (size_t)rn((int)blen + 1) : 0; + tt_write(bulk, buf, split); + tt_write(bulk, buf + split, blen - split); + for (size_t i = 0; i < blen; i++) + tt_write(bytes, buf + i, 1); + assert_same(bulk, bytes, seed, step); + if (rn(4) == 0) { + cols = 4 + rn(30); + rows = 2 + rn(10); + bool resized = tt_resize(bulk, cols, rows); + resized &= tt_resize(bytes, cols, rows); + assert(resized); + assert_same(bulk, bytes, seed, step); + } + } + term_snapshot_free(snap_bulk); + term_snapshot_free(snap_bytes); + term_free(bulk); + term_free(bytes); + } + return 0; +} diff --git a/tools/amalgamate.py b/tools/amalgamate.py new file mode 100644 index 0000000..f2cdf3b --- /dev/null +++ b/tools/amalgamate.py @@ -0,0 +1,94 @@ +#!/usr/bin/env python3 +"""Write the term core as one term.c and one term.h. + +term.h joins the public headers. term.c joins every source file and puts +each private header in at its first include. A host compiles term.c once, +with any C11 compiler, and links libm. + + amalgamate.py TERM_DIR OUT_DIR [DEPFILE] + +TERM_DIR is the directory with src/ and include/. DEPFILE lists every file +read, for the build system. +""" + +import re +import sys +from pathlib import Path + +INCLUDE = re.compile(r'^\s*#\s*include\s+"([^"]+)"') +PUBLIC = ("term_types.h", "term_input.h", "term.h") + + +class Joiner: + def __init__(self, root: Path): + self.root = root + # The sources find the public headers without the term/ prefix. + self.search = (root / "src", root / "include" / "term") + self.seen: set[Path] = set() + + def resolve(self, name: str, base: Path) -> Path: + for d in (base,) + self.search: + p = (d / name).resolve() + if p.is_file(): + return p + raise SystemExit(f"{base}: cannot find {name}") + + def emit(self, path: Path, out: list[str]) -> None: + """Copy path to out. Replace each local include with the file, + the first time only.""" + self.seen.add(path) + rel = path.relative_to(self.root) + out.append(f"/* ---- {rel} ---- */\n") + for line in path.read_text().splitlines(keepends=True): + m = INCLUDE.match(line) + if not m: + out.append(line) + continue + dep = self.resolve(m.group(1), path.parent) + if dep not in self.seen: + self.emit(dep, out) + out.append(f"/* ---- end {rel} ---- */\n") + + +def sources(src: Path) -> list[Path]: + files = sorted(src.glob("*.c")) + sorted((src / "osc").glob("*.c")) + return [f.resolve() for f in files] + + +def main() -> int: + if len(sys.argv) not in (3, 4): + print(__doc__, file=sys.stderr) + return 2 + root = Path(sys.argv[1]).resolve() + public = root / "include" / "term" + out_dir = Path(sys.argv[2]) + out_dir.mkdir(parents=True, exist_ok=True) + banner = "/* Generated by tools/amalgamate.py. Do not edit. */\n" + + header = Joiner(root) + text = [banner] + header.emit(public / "term.h", text) + (out_dir / "term.h").write_text("".join(text)) + + body = Joiner(root) + text = [banner, + "/* The private headers have helpers that only some platforms or\n" + " * callers outside the core use. Clang warns about them here. */\n" + "#if defined(__clang__)\n" + '#pragma clang diagnostic ignored "-Wunused-function"\n' + "#endif\n", + '#include "term.h"\n'] + # The public headers are in term.h, so the sources skip them. + body.seen.update((public / name).resolve() for name in PUBLIC) + for src in sources(root / "src"): + body.emit(src, text) + (out_dir / "term.c").write_text("".join(text)) + if len(sys.argv) == 4: + read = sorted(header.seen | body.seen) + deps = " ".join(str(p).replace(" ", "\\ ") for p in read) + Path(sys.argv[3]).write_text(f"{out_dir / 'term.c'}: {deps}\n") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/generate_unicode.py b/tools/generate_unicode.py new file mode 100644 index 0000000..cf72b87 --- /dev/null +++ b/tools/generate_unicode.py @@ -0,0 +1,117 @@ +#!/usr/bin/env python3 + +import argparse +import re +import sys +from pathlib import Path + +VERSION_PATTERNS = ( + re.compile(r"^#\s*[A-Za-z-]+-(\d+\.\d+(?:\.\d+)?)\.txt\s*$"), + re.compile(r"^#\s*Version:\s*(\d+\.\d+(?:\.\d+)?)\s*$"), + re.compile(r"^#\s*Used with Emoji Version (\d+\.\d+(?:\.\d+)?)\b"), +) + +def merge(values): + values = sorted(set(values)) + ranges = [] + for cp in values: + if ranges and cp == ranges[-1][1] + 1: + ranges[-1] = (ranges[-1][0], cp) + else: + ranges.append((cp, cp)) + return ranges + +def field_ranges(path, wanted): + values = [] + for raw in path.read_text().splitlines(): + line = raw.split("#", 1)[0].strip() + if not line: + continue + span, prop = (part.strip() for part in line.split(";", 1)) + if prop not in wanted: + continue + ends = [int(x, 16) for x in span.split("..")] + values.extend(range(ends[0], ends[-1] + 1)) + return merge(values) + +def normalize_version(version): + parts = version.split(".") + return ".".join(parts + ["0"] * (3 - len(parts))) + +def header_version(path): + found = set() + for raw in path.read_text().splitlines(): + if not raw.startswith("#"): + break + for pattern in VERSION_PATTERNS: + match = pattern.match(raw) + if match: + found.add(normalize_version(match.group(1))) + if len(found) != 1: + sys.exit(f"{path}: expected one Unicode version in the header, found {sorted(found) or 'none'}") + return found.pop() + +def unicode_categories(path): + by_category = {} + pending = None + for raw in path.read_text().splitlines(): + fields = raw.split(";") + cp, name, category = int(fields[0], 16), fields[1], fields[2] + if name.endswith(", First>"): + pending = (cp, category) + continue + if name.endswith(", Last>"): + start, first_category = pending + assert category == first_category + by_category.setdefault(category, []).extend(range(start, cp + 1)) + pending = None + else: + by_category.setdefault(category, []).append(cp) + assert pending is None + return by_category + +def emit_table(out, name, ranges): + out.append(f"static const struct unicode_range {name}[] = {{") + for i in range(0, len(ranges), 4): + chunk = ranges[i:i + 4] + out.append("\t" + ",".join(f"{{0x{lo:X},0x{hi:X}}}" for lo, hi in chunk) + ",") + out.append("};\n") + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("unicode_data", type=Path) + parser.add_argument("east_asian_width", type=Path) + parser.add_argument("emoji_data", type=Path) + parser.add_argument("output", type=Path) + args = parser.parse_args() + + versions = {path: header_version(path) for path in (args.east_asian_width, args.emoji_data)} + if len(set(versions.values())) != 1: + sys.exit("input files disagree on the Unicode version: " + + ", ".join(f"{path}={version}" for path, version in versions.items())) + version = next(iter(versions.values())) + + categories = unicode_categories(args.unicode_data) + assigned = {cp for cps in categories.values() for cp in cps} + # Conjoining Hangul vowels and finals draw inside the leading consonant. + jamo = list(range(0x1160, 0x1200)) + [cp for cp in range(0xD7B0, 0xD800) if cp in assigned] + zero = merge(categories.get("Mn", []) + categories.get("Me", []) + + categories.get("Cf", []) + jamo) + nonprint = merge(categories.get("Cc", []) + categories.get("Cs", [])) + wide = field_ranges(args.east_asian_width, {"W", "F"}) + spacing = merge(categories.get("Mc", [])) + pictographic = field_ranges(args.emoji_data, {"Extended_Pictographic"}) + emoji_presentation = field_ranges(args.emoji_data, {"Emoji_Presentation"}) + + out = [f"/* Generated from Unicode {version} data by tools/generate_unicode.py. */", "", + f'#define UNICODE_DATA_VERSION "{version}"', ""] + emit_table(out, "zero_width", zero) + emit_table(out, "nonprintable", nonprint) + emit_table(out, "wide", wide) + emit_table(out, "spacing_mark", spacing) + emit_table(out, "extended_pictographic", pictographic) + emit_table(out, "emoji_presentation", emoji_presentation) + args.output.write_text("\n".join(out)) + +if __name__ == "__main__": + main() |