Watch
1
0
Fork
You've already forked souveraine
0
souveraine/assets/face/index.html

314 lines
14 KiB
HTML
Raw Normal View History

<!doctype html>
<html>
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1,user-scalable=no">
<style>
html, body {
margin: 0; padding: 0; height: 100%; width: 100%;
background: transparent; overflow: hidden;
-webkit-user-select: none; user-select: none;
}
#live2d-widget { position: fixed; inset: 0; }
#live2d { width: 100%; height: 100%; }
#bubble {
position: fixed; left: 8px; right: 8px; top: 12px;
padding: 10px 14px; border-radius: 14px;
background: rgba(20,20,24,0.82); color: #f2f2f2;
font: 15px/1.35 system-ui, sans-serif;
opacity: 0; transition: opacity .2s ease;
}
#bubble.up { opacity: 1; }
</style>
</head>
<body>
<div id="live2d-widget"><canvas id="live2d"></canvas></div>
<div id="bubble"></div>
her face on the glass: it draws, it listens, and it gets out of the way The rig never drew. live2d.js is not only the Cubism runtime — it carries the reference's character layer, which reads #live_talk every update as its motion-state slot. Our page omitted the element, so it threw on the first frame after the model and all four textures had loaded: assets 200, WebGL live, no error reaching Rust, blank canvas. --app-id was a no-op. set_prgname ran after GTK had already created the surface, so the compositor saw souveraine-web and window rules could only match the title. It has to run before the event loop. Press and hold her to speak: pw-record at 16k mono s16, release transcribes through souveraine-stt --file and goes out over Souveraine.send, so the sidebar logs it. Whisper wraps with newlines and the transcript arrives through a SplitParser, so it is collapsed here or one utterance becomes four turns. Double tap dismisses; the clock and quote fade out when she is up and the clock stops being a target, because a double tap on an invisible object is not a way back. The page reports its silhouette and the host installs it as the input region. Claiming the GL context for preserveDrawingBuffer must happen after the canvas is sized — before it, the context is born 300x150 and she renders as a thumbnail in the corner. Console and errors are forwarded to the host's stdout. Without that a rig that fails to draw is silent in every direction, which is what cost this the afternoon.
2026-08-07 00:24:12 -04:00
<!--
Not decoration, and not optional: `live2d.js` is not only the Cubism
runtime. It is the reference's webpack bundle, character layer included, and
`Live2DModel.update()` reads this element's value on **every frame** as its
motion-state slot — "1" starts a talk motion, "0" resets to idle and then
writes back "3". The reference supplies it from `_includes/live2d.html`; a
page that omits it throws
TypeError: null is not an object
(evaluating 'document.getElementById("live_talk").value')
on the first update, after the runtime has already logged its banner and
loaded the model and all four textures. So everything looks healthy —
assets 200, WebGL live, requestAnimationFrame running at 60 — and the rig
draws nothing at all. That is exactly the "canvas is there and nothing
draws" TASK-59 predicted would cost an afternoon, and it did (2026-08-06).
The runtime writes to this too, so it must be a real input, not a constant.
-->
<input type="hidden" id="live_talk" value="1">
<script src="live2d/js/live2d.js"></script>
<script>
// The character layer, reduced to what the shell drives. The reference's
// message.js owns idle chatter, a talk box and sessionStorage position; none of
// that belongs here — the shell owns the conversation and the summoning, and
// idle chatter that is hers comes from the subconscious, not a random line
// table (TASK-59 Q2).
(function () {
var canvas = document.getElementById('live2d');
var bubble = document.getElementById('bubble');
var model = null;
function fit() {
canvas.width = window.innerWidth * window.devicePixelRatio;
canvas.height = window.innerHeight * window.devicePixelRatio;
}
window.addEventListener('resize', fit);
fit();
her face on the glass: it draws, it listens, and it gets out of the way The rig never drew. live2d.js is not only the Cubism runtime — it carries the reference's character layer, which reads #live_talk every update as its motion-state slot. Our page omitted the element, so it threw on the first frame after the model and all four textures had loaded: assets 200, WebGL live, no error reaching Rust, blank canvas. --app-id was a no-op. set_prgname ran after GTK had already created the surface, so the compositor saw souveraine-web and window rules could only match the title. It has to run before the event loop. Press and hold her to speak: pw-record at 16k mono s16, release transcribes through souveraine-stt --file and goes out over Souveraine.send, so the sidebar logs it. Whisper wraps with newlines and the transcript arrives through a SplitParser, so it is collapsed here or one utterance becomes four turns. Double tap dismisses; the clock and quote fade out when she is up and the clock stops being a target, because a double tap on an invisible object is not a way back. The page reports its silhouette and the host installs it as the input region. Claiming the GL context for preserveDrawingBuffer must happen after the canvas is sized — before it, the context is born 300x150 and she renders as a thumbnail in the corner. Console and errors are forwarded to the host's stdout. Without that a rig that fails to draw is silent in every direction, which is what cost this the afternoon.
2026-08-07 00:24:12 -04:00
// Claim the context before the runtime does, only to set one flag — and
// strictly **after** `fit()`, which is not a detail.
//
// `getContext` returns the existing context on every later call and ignores
// the attributes, so whoever asks first decides, and the runtime asks
// without `preserveDrawingBuffer`. Without that flag the drawing buffer is
// cleared the moment the frame is composited, which is before any callback
// can read it — so sampling her silhouette gets a blank image and the input
// region that depends on it collapses to a stray rectangle.
//
// Asking *before* `fit()` breaks her instead, and spectacularly: the canvas
// is still at its default 300x150 then, so the context is born that size and
// `gl.viewport` stays there while `fit()` grows the drawing buffer to the
// panel. The rig then draws into a 300x150 corner of a 1080x1994 buffer —
// and because GL's origin is bottom-left, she appears as a thumbnail in the
// bottom-left of the screen. Casey, 2026-08-06: "been getting a tiny version
// that's scaled odd". Same cause as the silhouette reading 9 lit cells out
// of 864: there really was almost nothing there.
try {
canvas.getContext('webgl', { preserveDrawingBuffer: true, alpha: true });
} catch (e) {}
// loadlive2d is live2d.js's entry point: canvas id, model.json url.
try {
loadlive2d('live2d', 'live2d/model/histoire/model.json');
} catch (e) {
console.error('rig failed to load', e);
}
var hideAt = null;
window.face = {
// One line of her speech. Called repeatedly as tokens stream, so it
// replaces rather than appends — the shell holds the accumulated text.
say: function (text) {
bubble.textContent = text;
bubble.classList.add('up');
hideAt = Date.now() + 6000;
},
clear: function () { bubble.classList.remove('up'); },
// Posture from the shell. The rig's own motion groups are named by the
// reference (idle, tap); anything richer waits on her own rig.
posture: function (name) {
if (typeof window.live2dMotion === 'function') window.live2dMotion(name);
},
// Look at a point on the panel, in her window's own CSS pixels.
//
// Dispatched as a synthetic pointer move on the canvas rather than driven
// through the rig's parameters by hand: `live2d.js` already binds
// mousemove/touchmove and already turns a pointer into head angle and eye
// direction with its own damping. Feeding its path costs nothing and
// cannot drift from what a real finger does, because it *is* what a real
// finger does.
//
// Synthetic events are why this works at all now. Her input region is her
// silhouette, so the compositor stops handing her contacts that land on
// the wallpaper — which is what made her stop tracking unless you touched
// her first. These do not go through the compositor, so she can follow a
// finger she is deliberately not being given.
lookAt: function (x, y) {
try {
canvas.dispatchEvent(new MouseEvent('mousemove', {
clientX: x, clientY: y, bubbles: true
}));
} catch (e) {}
},
// Nobody is touching the glass. Without this she holds the last position
// she was told about and stares at it until someone touches the screen
// again.
lookAway: function () {
try {
canvas.dispatchEvent(new MouseEvent('mouseout', { bubbles: true }));
} catch (e) {}
}
};
setInterval(function () {
if (hideAt && Date.now() > hideAt) { hideAt = null; window.face.clear(); }
}, 500);
her face on the glass: it draws, it listens, and it gets out of the way The rig never drew. live2d.js is not only the Cubism runtime — it carries the reference's character layer, which reads #live_talk every update as its motion-state slot. Our page omitted the element, so it threw on the first frame after the model and all four textures had loaded: assets 200, WebGL live, no error reaching Rust, blank canvas. --app-id was a no-op. set_prgname ran after GTK had already created the surface, so the compositor saw souveraine-web and window rules could only match the title. It has to run before the event loop. Press and hold her to speak: pw-record at 16k mono s16, release transcribes through souveraine-stt --file and goes out over Souveraine.send, so the sidebar logs it. Whisper wraps with newlines and the transcript arrives through a SplitParser, so it is collapsed here or one utterance becomes four turns. Double tap dismisses; the clock and quote fade out when she is up and the clock stops being a target, because a double tap on an invisible object is not a way back. The page reports its silhouette and the host installs it as the input region. Claiming the GL context for preserveDrawingBuffer must happen after the canvas is sized — before it, the context is born 300x150 and she renders as a thumbnail in the corner. Console and errors are forwarded to the host's stdout. Without that a rig that fails to draw is silent in every direction, which is what cost this the afternoon.
2026-08-07 00:24:12 -04:00
// Press and hold to speak; a short press is still a tap.
//
// This surface has no keyboard and is not going to get one — Casey,
// 2026-08-06: "input is audio only on this interface, the sidepanel is the
// text interface." So the whole of talking to her is one gesture, and the
// page only reports its edges: the shell owns the recorder, the endpoint
// and the transcript, the same way it already owns the conversation.
//
// The threshold separates the two meanings of touching her rather than
// giving the tap a modifier. Below it she plays a motion; above it she
// listens. `pointercancel` matters as much as `pointerup`: a hold that
// becomes a drag, or a finger that leaves the surface, must stop the
// recorder — a listener nobody closed is the failure this gesture can
// actually cause.
var HOLD_MS = 350;
var holdTimer = null;
var listening = false;
function endHold(send) {
if (holdTimer) { clearTimeout(holdTimer); holdTimer = null; }
if (!listening) return false;
listening = false;
if (window.souveraine) window.souveraine.talk(send ? 'end' : 'cancel');
return true;
}
canvas.addEventListener('pointerdown', function (e) {
if (holdTimer) clearTimeout(holdTimer);
holdTimer = setTimeout(function () {
holdTimer = null;
listening = true;
if (window.souveraine) window.souveraine.talk('start');
}, HOLD_MS);
// So a finger that slides off still delivers its release here.
try { canvas.setPointerCapture(e.pointerId); } catch (err) {}
});
// Two quick taps send her away, mirroring the clock's own double tap that
// brought her here. Same window as the clock uses (350ms), so the gesture
// means one thing on this device rather than two.
var DOUBLE_MS = 350;
var lastTapAt = -1;
canvas.addEventListener('pointerup', function () {
// A hold that reached the threshold was speech, not a tap, and must not
// also count toward a double tap — releasing after speaking would
// otherwise be half of a dismissal.
if (endHold(true)) { lastTapAt = -1; return; }
if (!window.souveraine) return;
var now = Date.now();
if (lastTapAt > 0 && now - lastTapAt <= DOUBLE_MS) {
lastTapAt = -1;
window.souveraine.dismiss();
return;
}
lastTapAt = now;
// Single taps are hers to answer with a motion.
window.souveraine.tapped('body');
});
her face on the glass: it draws, it listens, and it gets out of the way The rig never drew. live2d.js is not only the Cubism runtime — it carries the reference's character layer, which reads #live_talk every update as its motion-state slot. Our page omitted the element, so it threw on the first frame after the model and all four textures had loaded: assets 200, WebGL live, no error reaching Rust, blank canvas. --app-id was a no-op. set_prgname ran after GTK had already created the surface, so the compositor saw souveraine-web and window rules could only match the title. It has to run before the event loop. Press and hold her to speak: pw-record at 16k mono s16, release transcribes through souveraine-stt --file and goes out over Souveraine.send, so the sidebar logs it. Whisper wraps with newlines and the transcript arrives through a SplitParser, so it is collapsed here or one utterance becomes four turns. Double tap dismisses; the clock and quote fade out when she is up and the clock stops being a target, because a double tap on an invisible object is not a way back. The page reports its silhouette and the host installs it as the input region. Claiming the GL context for preserveDrawingBuffer must happen after the canvas is sized — before it, the context is born 300x150 and she renders as a thumbnail in the corner. Console and errors are forwarded to the host's stdout. Without that a rig that fails to draw is silent in every direction, which is what cost this the afternoon.
2026-08-07 00:24:12 -04:00
canvas.addEventListener('pointercancel', function () { endHold(false); });
window.addEventListener('blur', function () { endHold(false); });
// ## Her silhouette is where she takes touches; the rest is the home screen
//
// A transparent window is still a rectangle, and hers is the full width of
// the panel — so without this every tap meant for a widget behind her was
// swallowed by empty glass. The compositor cannot work the shape out: only
// this page knows which pixels she actually drew.
//
// Sampled from her own alpha rather than guessed as a box over her body,
// because she is a seated figure with gaps that matter — between her arms
// and her sides, under the book, either side of her hair — and a bounding
// box would hand all of them to her.
//
// Coarse on purpose. A grid cell is tens of pixels, the mask is the union of
// the cells that contain anything, and that costs one small readback instead
// of a per-pixel region the compositor would have to intersect on every
// contact. Erring outward by half a cell is the right direction: it is
// better to take a tap just off her elbow than to drop one on it.
var GRID_X = 24, GRID_Y = 36;
var shapeCanvas = document.createElement('canvas');
shapeCanvas.width = GRID_X;
shapeCanvas.height = GRID_Y;
var shapeCtx = shapeCanvas.getContext('2d', { willReadFrequently: true });
var lastShape = '';
// The first few attempts report unconditionally, including failures. An
// input region that silently never publishes looks exactly like one that
// published correctly — the glass is identical either way — so the quiet
// path is the one that needs a voice. Bounded, because after that it is
// just noise on every sample.
var shapeReports = 0;
function publishShape() {
if (!window.souveraine || !canvas.width || !canvas.height) {
if (shapeReports++ < 3)
console.log('shape: skipped — souveraine=' + (!!window.souveraine)
+ ' canvas=' + canvas.width + 'x' + canvas.height
+ ' shape_fn=' + (window.souveraine && typeof window.souveraine.shape));
return;
}
var cw = window.innerWidth / GRID_X, ch = window.innerHeight / GRID_Y;
try {
// Scaled down by the GPU on the way in, so the readback is 24x36 and
// not the full panel. Must happen inside a frame: the WebGL canvas has
// no preserveDrawingBuffer, so its pixels are gone after compositing.
shapeCtx.clearRect(0, 0, GRID_X, GRID_Y);
shapeCtx.drawImage(canvas, 0, 0, GRID_X, GRID_Y);
var data = shapeCtx.getImageData(0, 0, GRID_X, GRID_Y).data;
} catch (e) {
return;
}
var rects = [];
for (var gy = 0; gy < GRID_Y; gy++) {
var runStart = -1;
for (var gx = 0; gx <= GRID_X; gx++) {
var solid = gx < GRID_X && data[(gy * GRID_X + gx) * 4 + 3] > 8;
if (solid && runStart < 0) runStart = gx;
// Runs merged along the row, so a full-width band is one rectangle
// instead of 24 — the region is unioned rect by rect on the other side.
if (!solid && runStart >= 0) {
rects.push([Math.floor(runStart * cw), Math.floor(gy * ch),
Math.ceil((gx - runStart) * cw), Math.ceil(ch)]);
runStart = -1;
}
}
}
// Nothing drawn yet: say nothing. An empty report means "all of it" to the
// host, and publishing that on every frame before the rig loads would make
// her briefly swallow the screen each time she is summoned.
if (rects.length === 0) {
if (shapeReports++ < 3) {
var solid = 0;
for (var i = 3; i < data.length; i += 4) if (data[i] > 8) solid++;
console.log('shape: nothing drawn — ' + solid + ' of ' + (GRID_X * GRID_Y)
+ ' cells had alpha; the sample read a blank buffer');
}
return;
}
var key = JSON.stringify(rects);
if (key === lastShape) return;
lastShape = key;
// Reported, because the shape is invisible on the glass and wrong is
// indistinguishable from right by looking: a single rectangle means the
// sampling read a blank buffer and she is swallowing the screen again.
if (shapeReports++ < 3) {
var lit = 0, rowsLit = 0;
for (var gy2 = 0; gy2 < GRID_Y; gy2++) {
var any = false;
for (var gx2 = 0; gx2 < GRID_X; gx2++)
if (data[(gy2 * GRID_X + gx2) * 4 + 3] > 8) { lit++; any = true; }
if (any) rowsLit++;
}
// Cells and rows, not just rectangles: a low rect count can mean either
// "she is a compact shape" or "the sample barely saw her", and only the
// coverage separates them.
console.log('shape: ' + rects.length + ' rects, ' + lit + '/' + (GRID_X * GRID_Y)
+ ' cells lit across ' + rowsLit + '/' + GRID_Y + ' rows');
}
window.souveraine.shape(rects);
}
// Her outline changes when she moves, but slowly and not by much, so this
// does not belong in the draw loop. Twice a second is well inside the time
// it takes to reach for something behind her, and it costs one 24x36
// readback.
setInterval(function () { requestAnimationFrame(publishShape); }, 500);
})();
</script>
</body>
</html>