her face on the glass: it draws, it listens, and it gets out of the way
The rig never drew. live2d.js is not only the Cubism runtime — it carries the reference's character layer, which reads #live_talk every update as its motion-state slot. Our page omitted the element, so it threw on the first frame after the model and all four textures had loaded: assets 200, WebGL live, no error reaching Rust, blank canvas. --app-id was a no-op. set_prgname ran after GTK had already created the surface, so the compositor saw souveraine-web and window rules could only match the title. It has to run before the event loop. Press and hold her to speak: pw-record at 16k mono s16, release transcribes through souveraine-stt --file and goes out over Souveraine.send, so the sidebar logs it. Whisper wraps with newlines and the transcript arrives through a SplitParser, so it is collapsed here or one utterance becomes four turns. Double tap dismisses; the clock and quote fade out when she is up and the clock stops being a target, because a double tap on an invisible object is not a way back. The page reports its silhouette and the host installs it as the input region. Claiming the GL context for preserveDrawingBuffer must happen after the canvas is sized — before it, the context is born 300x150 and she renders as a thumbnail in the corner. Console and errors are forwarded to the host's stdout. Without that a rig that fails to draw is silent in every direction, which is what cost this the afternoon.
This commit is contained in:
parent
78496e10e1
commit
ac35b3436b
6 changed files with 671 additions and 14 deletions
|
|
@ -120,6 +120,7 @@ modules/ii/appInventory/AppInventoryScope.qml souveraine/modules/ii/appInventory
|
|||
modules/ii/overview/Overview.qml souveraine/modules/ii/overview/Overview.qml
|
||||
modules/ii/overview/AppGrid.qml souveraine/modules/ii/overview/AppGrid.qml
|
||||
modules/ii/background/widgets/clock/CookieClock.qml souveraine/modules/ii/background/widgets/clock/CookieClock.qml
|
||||
modules/ii/background/widgets/clock/CookieQuote.qml souveraine/modules/ii/background/widgets/clock/CookieQuote.qml
|
||||
modules/ii/screenCorners/ScreenCorners.qml souveraine/modules/ii/screenCorners/ScreenCorners.qml
|
||||
modules/ii/onScreenKeyboard/OnScreenKeyboard.qml souveraine/modules/ii/onScreenKeyboard/OnScreenKeyboard.qml
|
||||
modules/common/Persistent.qml souveraine/modules/common/Persistent.qml
|
||||
|
|
|
|||
|
|
@ -233,11 +233,45 @@ Item {
|
|||
// found" while the double tap below worked fine, because the tap was the
|
||||
// first reference. The dial and the agent both need it addressable before
|
||||
// anyone has tapped anything.
|
||||
readonly property bool faceReady: Face.rigPresent
|
||||
readonly property bool faceUp: Face.joined
|
||||
|
||||
// She takes this spot, so the clock gets out of it.
|
||||
//
|
||||
// `Face.rigPresent` used to be read here for the same construct-the-
|
||||
// singleton reason and **no such property exists** — the pre-flight rig
|
||||
// check was deleted from `Face.qml` (a guard that answered false for a
|
||||
// directory that was plainly there) and this reference was left behind,
|
||||
// binding to `undefined` ever since. `joined` is real, and it is also the
|
||||
// thing worth watching, so the touch and the meaning are the same read.
|
||||
opacity: root.faceUp ? 0 : 1
|
||||
scale: root.faceUp ? 0.86 : 1
|
||||
|
||||
// Out faster than in. Arriving is her entrance and wants to be seen;
|
||||
// leaving is just the clock getting out of the way, and a slow fade there
|
||||
// reads as lag rather than as motion. The 0.86 shrink is the same value
|
||||
// the overview cards use for their intro, so the two pieces of theatre on
|
||||
// this device move alike rather than each inventing a number.
|
||||
Behavior on opacity {
|
||||
NumberAnimation {
|
||||
duration: root.faceUp ? 180 : 260
|
||||
easing.type: Easing.OutCubic
|
||||
}
|
||||
}
|
||||
Behavior on scale {
|
||||
NumberAnimation {
|
||||
duration: root.faceUp ? 180 : 260
|
||||
easing.type: Easing.OutCubic
|
||||
}
|
||||
}
|
||||
|
||||
MouseArea {
|
||||
anchors.fill: parent
|
||||
acceptedButtons: Qt.LeftButton
|
||||
// A faded clock is not a target. Without this the way back is a double
|
||||
// tap on an invisible object, which is not a way back — it is a thing
|
||||
// you have to already know. Dismissing her is a double tap on *her*
|
||||
// instead, which is the same gesture in the same place as summoning.
|
||||
enabled: !root.faceUp
|
||||
// Above the clock's children, not below. `z: -1` was the first guess
|
||||
// and it is why the first version did nothing: the clock's own visual
|
||||
// items sit at the default z, so the handler was underneath every one
|
||||
|
|
|
|||
|
|
@ -0,0 +1,89 @@
|
|||
// Souveraine fork of ii's CookieQuote. One change: it yields to her face.
|
||||
//
|
||||
// The quote is the clock's sibling inside `ClockWidget`'s Column, not its
|
||||
// child, so `CookieClock`'s own fade cannot reach it — and the parent that
|
||||
// holds both lives in `ii-base`, which never reaches the phone (the base tree
|
||||
// has drifted ~900 files; overrides are how a fix arrives). So the yield is
|
||||
// duplicated here rather than written once above them both. If a third
|
||||
// widget ever needs it, that is the moment to override `ClockWidget` instead
|
||||
// of adding a third copy of this binding.
|
||||
import qs.modules.common
|
||||
import qs.modules.common.widgets
|
||||
import qs.services
|
||||
import QtQuick
|
||||
import Qt5Compat.GraphicalEffects
|
||||
|
||||
Item {
|
||||
id: root
|
||||
|
||||
readonly property string quoteText: Config.options.background.widgets.clock.quote.text
|
||||
|
||||
implicitWidth: quoteBox.implicitWidth
|
||||
implicitHeight: quoteBox.implicitHeight
|
||||
|
||||
// Out of her way, on the same timings the clock uses, because they are one
|
||||
// piece of furniture to look at even though they are two items in a
|
||||
// column. Left behind, the quote sits across her chest — which is where it
|
||||
// was, and how this was noticed.
|
||||
readonly property bool faceUp: Face.joined
|
||||
opacity: root.faceUp ? 0 : 1
|
||||
scale: root.faceUp ? 0.86 : 1
|
||||
|
||||
Behavior on opacity {
|
||||
NumberAnimation {
|
||||
duration: root.faceUp ? 180 : 260
|
||||
easing.type: Easing.OutCubic
|
||||
}
|
||||
}
|
||||
Behavior on scale {
|
||||
NumberAnimation {
|
||||
duration: root.faceUp ? 180 : 260
|
||||
easing.type: Easing.OutCubic
|
||||
}
|
||||
}
|
||||
|
||||
DropShadow {
|
||||
source: quoteBox
|
||||
anchors.fill: quoteBox
|
||||
horizontalOffset: 0
|
||||
verticalOffset: 2
|
||||
radius: 12
|
||||
samples: radius * 2 + 1
|
||||
color: Appearance.colors.colShadow
|
||||
transparentBorder: true
|
||||
}
|
||||
|
||||
Rectangle {
|
||||
id: quoteBox
|
||||
|
||||
implicitWidth: quoteRow.implicitWidth + 8 * 2
|
||||
implicitHeight: quoteRow.implicitHeight + 4 * 2
|
||||
radius: Appearance.rounding.small
|
||||
color: Appearance.colors.colSecondaryContainer
|
||||
|
||||
Row {
|
||||
id: quoteRow
|
||||
anchors.centerIn: parent
|
||||
spacing: 4
|
||||
|
||||
MaterialSymbol {
|
||||
id: quoteIcon
|
||||
anchors.top: parent.top
|
||||
iconSize: Appearance.font.pixelSize.huge
|
||||
text: "format_quote"
|
||||
color: Appearance.colors.colOnSecondaryContainer
|
||||
}
|
||||
StyledText {
|
||||
id: quoteStyledText
|
||||
horizontalAlignment: Text.AlignLeft
|
||||
text: Config.options.background.widgets.clock.quote.text
|
||||
color: Appearance.colors.colOnSecondaryContainer
|
||||
font {
|
||||
family: Appearance.font.family.reading
|
||||
pixelSize: Appearance.font.pixelSize.large
|
||||
weight: Font.Normal
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -79,13 +79,36 @@ Singleton {
|
|||
root.join();
|
||||
}
|
||||
|
||||
// How big she is on the glass.
|
||||
//
|
||||
// She has to be **smaller than the usable zone**, and that is the whole
|
||||
// reason this is set rather than left to the host's default. The
|
||||
// compositor stands furniture up centred and lifted off the bottom edge —
|
||||
// but it can only lift her into slack that exists, and at the default
|
||||
// 540x960 she measured 997 tall against a 931 zone (GTK adds ~37px of
|
||||
// titlebar even with decorations off), so she overflowed, had no slack,
|
||||
// and sat pinned at the top with nowhere to go. Under the zone, and the
|
||||
// lift is what puts her up by the clock instead of down on the dock.
|
||||
// Her size on the glass, and the canvas is not a crop — it is her scale.
|
||||
//
|
||||
// The Cubism view fits the rig to the canvas, so shrinking the canvas
|
||||
// shrinks *her*; it does not trim the empty margin around her. Tried on
|
||||
// 2026-08-06: 640 to cut the ~100px of dead space under her feet, and it
|
||||
// came back "a tiny version that's scaled odd" because the whole figure
|
||||
// came down with it. The dead space is the rig's own layout (`center_y`
|
||||
// and `width` in model.json), and moving it is a rig change, not a window
|
||||
// one. 760 is the size that reads right.
|
||||
readonly property string faceSize: "540x760"
|
||||
|
||||
Process {
|
||||
id: host
|
||||
command: ["souveraine-web",
|
||||
"--rig", root.rigDir,
|
||||
"--ipc", root.socketPath,
|
||||
"--transparent",
|
||||
"--app-id", "org.souveraine.face"]
|
||||
"--size", root.faceSize,
|
||||
"--app-id", "org.souveraine.face",
|
||||
"--title", "Ani"]
|
||||
stdout: SplitParser {
|
||||
splitMarker: "\n"
|
||||
onRead: line => {
|
||||
|
|
@ -100,6 +123,12 @@ Singleton {
|
|||
Souveraine.send(msg.text);
|
||||
else if (msg.event === "tapped")
|
||||
root.tapped(msg.area ?? "body");
|
||||
else if (msg.event === "talk")
|
||||
root._talk(msg.phase);
|
||||
else if (msg.event === "dismiss")
|
||||
root.leave();
|
||||
else if (msg.event === "console")
|
||||
root._pageSaid(msg.level, msg.text);
|
||||
}
|
||||
}
|
||||
onExited: root.joined = false
|
||||
|
|
@ -107,6 +136,118 @@ Singleton {
|
|||
|
||||
signal tapped(string area)
|
||||
|
||||
// Whether she is recording right now. One flag, so a second press cannot
|
||||
// start a second recorder over the first one's WAV.
|
||||
property bool listening: false
|
||||
|
||||
// ## Holding her to speak
|
||||
//
|
||||
// Press and hold, speak, release: the utterance goes through the existing
|
||||
// STT, the answer through the existing turn, and the sidebar logs both.
|
||||
// She is the *face* of the voice pipeline, not a second chat surface
|
||||
// (TASK-59 Q1a) — which is why nothing here holds a transcript or a
|
||||
// conversation, it only hands text to `Souveraine.send()`.
|
||||
//
|
||||
// The recorder is `pw-record` at 16k mono s16 and the transcription is
|
||||
// `souveraine-stt --file`, deliberately: that script already owns the
|
||||
// endpoint from Settings → Speech and the whole error vocabulary
|
||||
// (unreachable / 5xx / rejected), and a second copy of that here would be
|
||||
// the second answer to "where does dictation go". 16k mono s16 is not a
|
||||
// preference either — the comment in that script records that the server
|
||||
// 500s on anything else.
|
||||
//
|
||||
// Written as one `sh -c` rather than a helper on PATH because the shell
|
||||
// tree deploys as a unit and a new file on the device would need a package
|
||||
// to reach it (CLAUDE.md's rule, and the trap that left sessiond five days
|
||||
// stale). Two commands, one place.
|
||||
function _talk(phase) {
|
||||
if (phase === "start") {
|
||||
if (root.listening)
|
||||
return;
|
||||
root.listening = true;
|
||||
root._eval(`window.face.posture("listening")`);
|
||||
recorder.command = ["sh", "-c",
|
||||
"rm -f \"$W\"; pw-record --rate 16000 --channels 1 --format s16 \"$W\" & echo $! > \"$P\"; wait"];
|
||||
recorder.running = true;
|
||||
return;
|
||||
}
|
||||
if (!root.listening)
|
||||
return;
|
||||
root.listening = false;
|
||||
recorder.running = false;
|
||||
// Cancelled — a finger that slid off her, or the window losing focus.
|
||||
// The recording is dropped rather than transcribed: sending whatever
|
||||
// was captured before an abandoned gesture would put words she never
|
||||
// finished into the conversation.
|
||||
if (phase !== "end") {
|
||||
stopper.command = ["sh", "-c", `kill -INT $(cat "$P" 2>/dev/null) 2>/dev/null; rm -f "$P" "$W"`];
|
||||
stopper.running = true;
|
||||
root._eval(`window.face.posture("idle")`);
|
||||
return;
|
||||
}
|
||||
// The 0.3s is not padding: pw-record finalises the WAV header on the
|
||||
// way out, and reading it sooner gets a file the server rejects.
|
||||
// `souveraine-stt` learned this the same way and its comment says so.
|
||||
// The `tr`/`sed` is not tidying. Whisper wraps its output with
|
||||
// embedded newlines, and the transcript arrives here through a
|
||||
// `SplitParser` on "\n" — so four wrapped lines would be **four
|
||||
// separate messages** sent to her, one turn each, instead of one
|
||||
// utterance. `souveraine-stt` collapses them on its own typing leg
|
||||
// and says why in a comment; `--file` prints them raw, so the same
|
||||
// trap arrives by the other door and has to be closed on this side.
|
||||
transcriber.command = ["sh", "-c",
|
||||
`kill -INT $(cat "$P" 2>/dev/null) 2>/dev/null; rm -f "$P"; sleep 0.4; ` +
|
||||
`[ -s "$W" ] || exit 0; souveraine-stt --file "$W" ` +
|
||||
`| tr '\\n\\r' ' ' | sed 's/ */ /g; s/^ //; s/ $//'; echo; rm -f "$W"`];
|
||||
transcriber.running = true;
|
||||
root._eval(`window.face.posture("thinking")`);
|
||||
}
|
||||
|
||||
readonly property string _wav: Quickshell.env("XDG_RUNTIME_DIR") + "/souveraine-face-talk.wav"
|
||||
readonly property string _pid: Quickshell.env("XDG_RUNTIME_DIR") + "/souveraine-face-talk.pid"
|
||||
|
||||
Process {
|
||||
id: recorder
|
||||
environment: ({ W: root._wav, P: root._pid })
|
||||
}
|
||||
|
||||
Process {
|
||||
id: stopper
|
||||
environment: ({ W: root._wav, P: root._pid })
|
||||
}
|
||||
|
||||
Process {
|
||||
id: transcriber
|
||||
environment: ({ W: root._wav, P: root._pid })
|
||||
stdout: SplitParser {
|
||||
splitMarker: "\n"
|
||||
onRead: line => {
|
||||
const said = line.trim();
|
||||
if (said.length === 0)
|
||||
return;
|
||||
// Straight into the one transport, so the sidebar logs it and
|
||||
// the reply streams back to the bubble through the same
|
||||
// `onStreamEvent` her own speech already uses.
|
||||
Souveraine.send(said);
|
||||
}
|
||||
}
|
||||
onExited: root._eval(`window.face.posture("idle")`)
|
||||
}
|
||||
|
||||
// What the page says, where someone can see it.
|
||||
//
|
||||
// The host forwards console and errors on the same line protocol. Dropped
|
||||
// here, a rig that fails to draw is silent in every direction — which it
|
||||
// was, and it cost 2026-08-06 an afternoon: a missing `#live_talk` element
|
||||
// threw on the runtime's first update, after the model and all four
|
||||
// textures had loaded, so every other signal read healthy.
|
||||
function _pageSaid(level, text) {
|
||||
if (level === "error")
|
||||
console.warn("[face] page error:", text);
|
||||
else
|
||||
console.log("[face]", text);
|
||||
}
|
||||
|
||||
// Reachable by name, so the dial, a launcher and the agent all summon her
|
||||
// the same way rather than each growing a copy (TASK-30/31). `status`
|
||||
// answers rather than assumes — an agent that cannot ask whether she is up
|
||||
|
|
|
|||
Loading…
Reference in a new issue