Talon

From Giki

Contents

mishearings

what i said what talon understood possible fix/alias
series serious
urge each
or for/four
press plus
comma space colon space comgap,spamma: ", "
colon space comma space colp,spolon: ": "

alternatives

alphabet

alternatives for when the "defaults" (knausj-repo) are not well recognized merged with some from tavis rudds alphabet (https://gist.github.com/tavisrudd/5361092)

alphabet alternative
air acid
air aim
air alf
air any
air arch
air axel
bat batch
bat best
bat bill
bat bit
bat bob
bat bony
bat bot
cap carl
cap char
cap cock
cap cole
drum dan
drum dave
drum dozen
each birch
each ease
each east
each ebike
each ed
each eric
each eve
fine fake
fine faint
fine fig
fine flow
fine fox
fine foxy
fine fred
gust gary
gust gear
gust gig
gust golf
gust grace
harp hal
harp ham
harp hank
harp hate
harp hoop
harp hoof
harp hotel
sit dice
sit ice
sit ida
sit ink
sit ish
sit itchy
sit ivan
sit ivy
jury jack
jury jail
jury jane
jury jed
jury joe
jury judge
jury jug
jury julia
crunch kale
crunch keel
crunch kick
crunch kid
crunch kirk
crunch krife
crunch kyle
look else
look lazy
look little
look lou
look luka
made mary
made met
made midge
made mike
made miley
made murphy
near nasal
near nate
near neal
near nerd
near noy
odd oats
odd odin
odd oink
odd orange
odd osh
odd out
odd ox
odd oz
pit palm
pit pete
pit pink
pit pip
pit play
pit pom
pit premier
pit prince
quench quebec
quench queen
quench quinn
quench quick
red ralph
red ree
red remo
red ring
red rip
red risk
red robin
red root
sun salty
sun sam
sun side
sun sis
sun smith
sun soi
sun spun
trap tay
trap tea
trap tess
trap timy
trap tit
trap tosh
trap trip
urge unix
urge urch
urge urn
urge usher
urge ute
urge yuck
vest van
vest video
vest vince
vest vine
vest void
vest voss
vest you
whale wages
whale wasp
whale wes
whale wet
whale whip
whale will
whale wish
plex lex
plex x-ray
plex xin
yank yak
yank yan
yank yep
yank yoke
yank york
yank young
yank yugi
zip zap
zip zeke
zip zimeesi
zip zircon
zip zoko
zip zone
zip zoo
zip zulu

actions/commands/symbols

action/command/symbol knausj builtin alternative(s) user/custom alternative(s)
backspace clear left, delete, wipe junk
brace - (right) curve
enter - break, clap, oke, okay, pick, punch
exclamation mark (!) bang boom
colon - half stack
comma - drip
delete forward delete eat, junk right, slurp, spunk
dollar sign dollar dolly
hash - prison
katie - cathy
redo that - yup
tilde (~) - wiggle
tab - yes
undo that - crash, geez, fuck, nope, not that, rollback, scratch that, whoops, wrong
semicolon - sammy, semi, stack, wink
space - pah, put, void

formatters

knausj user/custom alternative(s)
all cap shout

Talon with remote Dragon

v2

Running Linux or Mac Talon against Windows Dragon, take two:
1. Find the latest “Released” message in 
beta Make sure to sort the search by recent instead of relevant.
2. Install Talon on Windows and run it (talon.exe or talon_console.exe). The system tray icon by the clock is barely visible with the default taskbar as it’s black, sorry, I will fix this sometime.
3. Right click on the Talon icon by the clock, go to Scripting, then “Open ~/talon”. Find draconity.toml in this directory and open it in a text editor.
4. Add this text at the end of draconity.toml:
[[socket]]
host = "0.0.0.0"
port = 38065
5. Copy the authentication key at the top of draconity.toml, you will need to have it to configure your other computer. Also note down the IP address of your Windows computer.
6. Install Talon on your non-Windows computer, open draconity.toml in the same way, edit the authentication key to match the Windows computer, and add this text at the end:
[[remote]]
host = "WINDOWS_IP_HERE"
port = 38065
7. Run Dragon on the Windows computer and wake up the microphone.

v1

source: https://news.ycombinator.com/item?id=24850417 

lunixbochs 4 months ago [–]

Talon can also use Dragon remotely from a Windows machine or VM, kind of like aenea but more tightly integrated (the host is driving instead of the guest).

And I do have some users voluntarily switching from Mac Dragon to the improved speech engine in the Talon beta. Mac Dragon has been kind of buggy for the last few years so you're not missing much.

	
	
ianhorn 4 months ago [–]

Any chance you have pointers on how to set that up? You'd probably laugh/cry to see my setup right now, with my Windows desktop on the left monitors and my MacBook on the right monitors because I need both... purely because Dragon is only sold on Windows since this started been an issue for me. A more tightly coupled super-aenea sounds pretty fantastic.

	
	
lunixbochs 4 months ago [–]

Sure: First run Talon on both sides. Then go to your talon home directory (click talon icon in tray -> scripting -> open ~/talon). There's a draconity.toml file there.

On the Dragon side, you need to uncomment and update the `[[socket]]` section to listen on an accessible IP.

On the client side (Mac in your case), uncomment / update the `[[remote]]` section to point at your other machine.

You also need to make sure both configs have the same secret value.

From there, restart Dragon and look in the Talon log on the Mac side for a line like "activating speech engine: Dragon".

To prevent command conflicts, I recommend setting Dragon to "command mode" (if you have DPI), and only adding scripts to Talon on the Mac side.

If it doesn't work, you can uncomment the `logfile` line in draconity.toml on the Dragon side, restart Dragon, and look in the log to see what's going on.

linux gnome remarks

also if you’re on latest gnome you should install appindicator support, they removed that for some reason

linux notifications

Put this in notify.py
from talon import Module, Context, app, speech_system, actions
ctx = Context()
mod = Module()
def on_phrase(j):
    phrase = getattr(j["parsed"], "_unmapped", j["phrase"])
    phrase = " ".join(word.split("\\")[0] for word in phrase)
    app.notify(phrase)
speech_system.register('post:phrase', on_phrase)
# above enables notifications. Below supports toggling them.
do_notify = True
@mod.action_class
class Actions:
    def notify_toggle():
        """Toggle OS notifications"""
        global do_notify
        if do_notify:
            speech_system.unregister('post:phrase', on_phrase)
        else:
            speech_system.register('post:phrase', on_phrase)
        do_notify = not do_notify
3:53
And in a .talon file: talon notify: user.notify_toggle().

linux pull all repos in ~/.talon/user

source: https://talonvoice.slack.com/archives/CKYAFAADA/p1651807016464379 

find ~/.talon/user -name .git -print -execdir git pull --ff-only \;

automation / run on start / app switch

load things on talon startup

from talon import app, speech_system

def on_launch():
    speech_system.mimic("command history")

app.register('launch', on_launch)

start talon in sleep mode

from talon import actions, app

def on_ready():
    actions.speech.disable()

app.register('launch', on_ready)

start talon with mouse control enabled

from talon import actions, app

def on_launch():
    actions.tracking.control_toggle()

app.register('launch', on_launch)

run action on app switch

def on_app_switch(app):
    ...
ui.register("app_activate", on_app_switch)
ui.register("app_launch", on_app_switch)

alternative paste method

in case anyone else is finding the actions.user.paste() problematic (too slow) on linux, you can use this replacement code in code/edit.py in the interim, which seems to be work fine based on testing (not sure if someone posted another workaround already): 

@mod.action_class
class Actions:
    def paste(text: str):
        """Pastes text and preserves clipboard"""
        #with clip.revert():
        #    clip.set_text(text)
        #    actions.edit.paste()
        #    # sleep here so that clip.revert doesn't revert the clipboard too soon
        #    actions.sleep("150ms")
        # TODO: qt clipboard is bugged, do it manually
        old = subprocess.check_output(['xclip', '-o', '-sel', 'clip'])
        clip.set_text(text)
        actions.edit.paste()
        actions.sleep("150ms")
        clip.set_text(old.decode("utf-8"))

note you may have to install xclip depending on your distro (edited)

change to webengine (mac specific)

from talon import speech_system, Context
from talon.engines.w2l import WebW2lEngine, W2lEngine
from talon.engines.webspeech import WebSpeechEngine
w2l = W2lEngine(model='en_US-sconv-beta6', debug=True)
speech_system.add_engine(w2l)
webspeech = WebSpeechEngine()
speech_system.add_engine(webspeech)
# set the default engine
ctx = Context()
ctx.settings = {
    'speech.engine': 'wav2letter',
}

from talon.mac import applescript
import subprocess
if applescript.run('''
on webSpeechRunning()
    if application id "com.google.Chrome" is not running then return false
    tell application id "com.google.Chrome" to get (every window's every tab's URL as string) contains "http://localhost:7419/"
end webSpeechRunning

webSpeechRunning()
''') == 'false':
    subprocess.run(['/usr/local/bin/launch', '-bi', 'com.google.Chrome', 'http://localhost:7419'])
    subprocess.run(['/usr/local/bin/appswitch', '-hi', 'com.google.Chrome'])

draw image

from talon.canvas import Canvas
from talon.skia import Image
im1 = Image.from_file("image1.png") # current directory is ~/.talon
def draw(canvas):
    canvas.draw_image(im1, canvas.x, canvas.y)
canvas = Canvas(0, 0, 200, 200)
canvas.register('draw', draw)
canvas.freeze()

react on hiss

from talon import noise, ctrl, cron
import time
​
active_hiss = {"start": None, "cron": None}
HISS_MINIMUM_THRESHOLD = 0.25
​
def hiss_over_threshold():
    if not active_hiss["start"]:
        return False
​
    delta = time.time() - active_hiss["start"]
    trigger = delta > HISS_MINIMUM_THRESHOLD
    if delta:
        print("hisssing complete %0.2fs (trigger = %s)" % (delta, trigger))
​
    return trigger
​
def stop_hiss():
    if hiss_over_threshold():
        ctrl.mouse_click(0)
​
    if active_hiss["cron"]:
        cron.cancel(active_hiss["cron"])
        active_hiss["cron"] = None
​
    active_hiss["start"] = None
​
def start_hiss():
    if not active_hiss["start"]:
        active_hiss["start"] = time.time()
​
    active_hiss["cron"] = cron.interval("16ms", check_hiss)
​
def on_hiss(noise):
    if noise:
        start_hiss()
    else:
        stop_hiss()
​
def check_hiss():
    delta = float(time.time() - active_hiss["start"])
    should_click = delta > HISS_MINIMUM_THRESHOLD
​
    print("hisssing check: %0.2f s"% delta)
​
    if should_click:
        stop_hiss()
​
noise.register('hiss', on_hiss)

vs code syntax highlight extension

source: https://marketplace.visualstudio.com/items?itemName=mrob95.vscode-talonscript

german

add german dictation mode (using `vosk`)

source: https://github.com/mqnc/talon_german

updated repos/forks due to changes in 0.3

https://github.com/2d6/talon_german

install specific vosk version (libvosk.dll error 0x7f)

C:\Users\gpunktschmitz\AppData\Roaming\talon\.venv\Scripts>pip install vosk==0.3.21
Requirement already satisfied: vosk==0.3.21 in c:\users\gpunktschmitz\appdata\roaming\talon\.venv\lib\site-packages (0.3.21)
Requirement already satisfied: cffi>=1.0 in c:\talon\lib\site-packages (from vosk==0.3.21) (1.15.1)
Requirement already satisfied: pycparser in c:\talon\lib\site-packages (from cffi>=1.0->vosk==0.3.21) (2.21)

knausj_german

https://github.com/hanuele/knausj_german

german.talon - switch between talon (en) and dragon (de)

source: https://github.com/wingerath/talon_german/blob/wolle_personal/german.talon 

mode: user.german
-

^(englisch | english)$:
    user.engine_mimic("geh schlafen")
    mode.disable("user.german")
    user.enable_withSound()

^(snore)$:
    user.engine_mimic("geh schlafen")
    user.sound_disable()
    user.app_color_black()


talon sleep | snore | go to sleep: skip()

^german$: user.sound_disable()

Umbruch: actions.key("shift-enter")

engine.py 

from talon import speech_system, Context, actions
from talon.engines.w2l import W2lEngine
from talon.engines.vosk import VoskEngine
#w2l = W2lEngine(model='en_US', debug=False)
w2l = W2lEngine(model='en_US-conformer', debug=False)
#vosk_de = VoskEngine(model='vosk-model-de-0.6', language='de_DE')
speech_system.add_engine(w2l)
#speech_system.add_engine(vosk_de)
# # ideally you integrate this into your engines.py file
#1
# from talon import speech_system, Context
#
#
# # especially this should not be here but in your engines.py file:
ctx = Context()
ctx.settings = {
    'speech.engine': 'wav2letter', # your default engine goes here
#    'speech.engine': 'dragon', # your default engine goes here
}

german webspeech mod

https://github.com/knausj85/knausj_talon/compare/master...jpaddison3:german-web-speech

script: print all talon commands

source: https://gist.github.com/tararoys/c538b7ae8e1f21db9a794c2c0f5becf4

script: reminder

source: https://gist.github.com/tararoys/d138b36731262fb8ae452814c13b9d4f

script: full mouse grid

https://github.com/tararoys/modified_full_mouse_grid

script: pokey cursorless

https://github.com/pokey/pokey_talon/tree/master/apps/vscode/cursorless

script: text selection

https://github.com/splondike/talon_telector

script: talon hud

https://github.com/chaosparrot/talon_hud

setup WebSpeech for Dictation

file: user/engines.py 

webspeech = WebSpeechEngine()
speech_system.add_engine(webspeech)

# set the default engine
ctx = Context()
ctx.settings = {
    'speech.engine': 'wav2letter',
}

file: user/webspeech.talon 

mode: dictation
-
settings():
    speech.engine = 'webspeech'
    speech.language = 'en_US'

While Talon is running, open http://localhost:7419 in Chrome. You'll probably also need to allow microphone access the first time you enable it.


practice

practice talon

https://chaosparrot.github.io/talon_practice/

Practice Speechrec Mappings

https://github.com/shervinemami/practice_speechrec_mappings

telector fix suggestion

https://github.com/splondike/talon_telector/blob/94381bf9a9d6bb7f513275a885fcdb0ee66b7917/talon_interface.py#L374

6:12
try replacing 391 - 404 with this:
src = (rect1.x, rect1.y + rect1.height / 2)
dst = (rect2.x + rect2.width+2, rect2.y + rect2.height / 2)
actions.mouse_move(*src)
actions.mouse_click()
time.sleep(0.1)
actions.mouse_move(*dst)
actions.key('shift:down')
actions.mouse_click()
actions.key('shift:up')
time.sleep(0.1)

knausj formatters actions

settings():
    user.code_private_function_formatter = "SNAKE_CASE"
    user.code_protected_function_formatter = "SNAKE_CASE"
    user.code_public_function_formatter = "SNAKE_CASE"
    user.code_private_variable_formatter = "SNAKE_CASE"
    user.code_protected_variable_formatter = "SNAKE_CASE"
    user.code_public_variable_formatter = "SNAKE_CASE"

code block example with optional formatters

if [<user.formatters>] <user.text> then:
    formatters = formatters or "NOOP"
    insert("if (")
    user.insert_formatted(text, formatters)
    insert(") {}")
    key(left)
    key(enter)

script for overlay commands

from talon.scripting import global_speech_system
from talon import canvas, ui, ctrl, cron
from talon.types import Rect
​
display_canvas = True
​
​
def _draw(canvas):
    print(canvas.x)
    paint = canvas.paint
    paint.textsize = 24
    canvas.clear("ffffff00")
    paint.color = "ffffffff"
    canvas.draw_text(text, canvas.x + 30, canvas.y + 30)
​
​
def reposition_canvas():
    x, y = ctrl.mouse_pos()
    print(x, y)
    can.move(x, y)
​
​
if display_canvas:
    can = canvas.Canvas.from_rect(Rect(500, 850, 600, 50))
    can.register("draw", _draw)
    can.show()
    can.freeze()
​
text = ""
# cron.interval("1s", reposition_canvas)
# reposition_canvas()
​
​
def _log(args):
    global text, can
    text = " ".join(args["text"])
    can.freeze()
​
​
if display_canvas:
    global_speech_system.register("phrase", _log)

script for hissing

from talon import noise, ctrl
def on_hiss(active):
    ctrl.mouse_click()
noise.register("hiss", on_hiss)

script cursor scrolling while hissing

from talon import ctrl, noise, cron
from time import sleep, time
from user.knausj_talon.code.mouse import start_cursor_scrolling, stop_scroll
​
start = 0
running = False
noise_length_threshold = "500ms"
threshold_passed = False
​
def still_running():
    global running
    global threshold_passed
    if running:
        threshold_passed = True
        start_cursor_scrolling()
        print('hiss duration passed threshold, starting gaze scroll')
​
def cursor_scroll_on_hiss(is_active):
  global start
  global running
  global threshold_passed
  if is_active:
    start = time()
    running = True
    cron.after(noise_length_threshold, still_running)
  else:
    running = False
    if threshold_passed:
        threshold_passed = False
        stop_scroll()
        print('end of hiss detected, disabling gaze scroll')
​
​
noise.register('hiss', cursor_scroll_on_hiss)

script simple cron task

from talon import cron
import time
def on_interval():
    now = time.perf_counter()
    print(f"{now:.3f}s")
cron.interval('1ms', on_interval)

script overwrite alphabet

"overwrite" knausj vocabulary with custom cope by adding context 

alphabet = "air bat cap drum each fine gust harp sit jury crunch look made near odd pit quench red sun trap urge vest whale plex yank zip"
letters = "abcdefghijklmnopqrstuvwxyz"
from talon import Context
ctx = Context()
ctx.matches = 'os: mac'
ctx.lists['user.letter'] = dict(zip(default_alphabet, letters_string))

script launch sound settings

one can type text by quoting it 

sound settings:
    key(super)
    "sound settings"
    key(enter)

script space to pause talon

key(space:down):
    speech.disable()
key(space:up):
    speech.enable()

script up_down_modifiers.talon

hold down <user.modifiers>+:
        key("{modifiers}:down")
release <user.modifiers>+:
        key("{modifiers}:up")

script: move mouse based on screenshot

source: https://github.com/thinium/knausj_talon/blob/bdf222f7/apps/ableton/locate.py 

from talon.experimental.locate import locate_hover
from talon import Module, ctrl
mod = Module()
@mod.action_class
class Actions:
    def locate(name: str):
        """Find an image on the screen and put the mouse in the center"""
        locate_hover(name, threshold=0.95)
    def nudge_mouse(x: int, y: int):
        """Move the mouse relatively"""
        _x, _y = ctrl.mouse_pos()
        ctrl.mouse_move(_x + x, _y + y)

script: windows lock screen

source: https://talonvoice.slack.com/archives/C7ENXA7C4/p1629413391497700?thread_ts=1629350122.432300&cid=C7ENXA7C4 

from talon import Module
mod = Module()

@mod.action_class
class Actions:
    def sleeplock():
        "Locks the screen"
        from ctypes import cdll

        user32lib = cdll.LoadLibrary(r'C:\Windows\System32\user32.dll')
        user32lib.LockWorkStation()

os: windows
-

lock it up:
    user.sleeplock()
    mimic('sleep all')

script: copy mouse position to clipboard

copy mouse (move|position):
    x = mouse_x()
    y = mouse_y()
    clip.set_text("mouse_move({x}, {y})")

vs code plugin cursorless

https://github.com/pokey/cursorless-talon

eye tracking

tobii eye tracker positioning

https://help.tobii.com/hc/en-us/articles/210250305-Position-in-front-of-the-Tobii-Eye-Tracker

[VIDEO] Talon Eye Tracking + Optikey Mouse

https://www.youtube.com/watch?v=PQkJE-rtn-g

calibration exits

if the tracker is very confused by what happened, calibration will exit before 9 dots have disappeared

-> rearrange angle / lights

taras awesome list

https://gist.github.com/tararoys/480d9b32c137d2f2f882ce06ccf12604

user scripts

alternative/additional software

OptiKey

OptiKey - Full computer control and speech with your eyes

VoiceAttack

https://voiceattack.com/

video showing how to use for programming

mac "bug": dots added randomly while typing

system preferences > keyboard

disable option "add period with double-space"

mac specific: key combinations

on Mac the modifier typically needs to be pressed "with the key" so one cannot

  1. press modifier key :down
  2. press key
  3. release modifier key :up

while on windows/linux this works

e.g. to press n then h while pressing cmd-ctrl: key(cmd-ctrl-n cmd-ctrl-h)

press key for 2 seconds

user.key_async("space", "2000ms")

videos

channels

VIM add-ins/programmes

programmes that enables to control the OS/application in a "vimmy"-way

Firefox

Chrome

Windows

Mac

select text and extend selection

when selecting text one would say select word left

to extend the selection one could say 2nd repetitive to extend the selection +1 word to the left (3rd, 4th, etc would do +2, +3 words, etc)

parrot

quickstart

  1. make ~/.talon/parrot
  2. copy parrot_ensemble_012.pkl to ~/.talon/parrot/model.pkl (from https://github.com/chaosparrot/parrotpy_tryout_bundle)
  3. create a ~/.talon/parrot/patterns.json file
  4. download parrot_integration.py from #beta and move it to ~/.talon/user/
  5. create a .talon file based on the pdf
  6. make the sounds

v2 changes

  1. updated parrot_integration.py
  2. update patterns.json: s/f1/f0/

A parrot testing file

source: https://talonvoice.slack.com/archives/C01QEDFH60J/p1627679500015900 

parrot(cluck):
        "cluck"
parrot(tut):
        "tut"
parrot(palate_click):
        "palate_click"
parrot(pop):
        "pop"
parrot(gluck):
        "gluck"
parrot(finger_snap):
        "finger_snap"
parrot(ch):
        "ch"
parrot(ah):
        "ah"
parrot(oh):
        "oh"
parrot(ee):
        "ee"
parrot(ue):
        "ue"
parrot(ae):
        "ae"
parrot(horse):
        "horse"
parrot(whistle):
        "whistle"
parrot(chopper):
        "chopper"
parrot(hurr):
        "hurr"
parrot(hmm):
        "hmm"
parrot(oo):
        "oo"
parrot(lll):
        "lll"
parrot(yee):
        "yee"
parrot(uh):
        "uh"
parrot(fff):
        "fff"
parrot(buzz)
        "buzz"
parrot(generator):
        "generator"
parrot(hiss):
        "hiss"
parrot(shush):
        "shush"
parrot(x):
        "x"

action after noise ends

parrot(hiss:stop): print("hiss stop")

action when noise starts

parrot(hiss:start): print("hiss start")

start/stop noise recognition

actions.mode.disable('noise')
actions.mode.enable('noise')

tryout bundle

https://github.com/chaosparrot/parrotpy_tryout_bundle

Parrot.PY

https://github.com/chaosparrot/parrot.py

video explaining how to: https://www.youtube.com/watch?v=2j90vhzRoT8

debugging

test command [REPL]

using the sim() function 

Talon REPL | Python 3.9.4 (default, Apr 24 2021, 16:58:24) [GCC 7.5.0] on linux)
>>> sim("slack close right sidebar")
[1] "slack close right sidebar"
   path: user/gpunkt_talon/mode-all.linux.talon
   rule: "slack close right sidebar"

show what events where triggered [REPL]

Talon REPL | Python 3.9.4 (default, Apr 24 2021, 16:58:24) [GCC 7.5.0] on linux)
>>> events.tail()

find where actions are defined/overloaded

Talon REPL | Python 3.9.4 (default, Apr 24 2021, 16:58:24) [GCC 7.5.0] on linux)
>>> actions.find('user.system_command')
[user/knausj_talon/core/system_command.py]
user.system_command(cmd: str)
  execute a command on the system

  * [user/knausj_talon/core/system_command.py]
      def system_command(cmd: str):
          """execute a command on the system"""
          os.system(cmd)

[user/knausj_talon/core/system_command.py]
user.system_command_nb(cmd: str)
  execute a command on the system without blocking

  * [user/knausj_talon/core/system_command.py]
      def system_command_nb(cmd: str):
          """execute a command on the system without blocking"""
          subprocess.Popen(cmd, shell=True)

search talon repos

https://search.talonvoice.com/

conformer

version history

  • a74 - first/original release
  • a108 - second release
  • b108 - third release: changed packaging but not performance

in Talon >= 0.2.0-8 conformer can be used without a language model (by renaming lm-ngram.bin)

alternative movement command

instead of go up fifth one can say go up five times

options

with b108 there is a torch option 

engine = W2lEngine(model='en_US-conformer', debug=True, torch=True)


mac key "option"

use 'alt' instead of 'opt' or 'option' - example: key(shift-option-down)

set speech timeout

save to settings.talon 

settings():
    speech.timeout = 0.3

customize for dictation mode

save to settings-dictation-mode.talon 

mode: dictation
-
settings():
    speech.timeout = 0.3

control zoom mouse scale

https://search.talonvoice.com/search/?q=zoom_mouse.config&fold_case=auto&regex=false&context=true

{number_small}

one can use numbers like key("cmd-alt-{number}") but if that number won't exceed 99 one should use {number_small} which caps at 99 whereas {number} is capable of infinite numbers (e.g. one hundred trillion)

mixed mode

source: https://talonvoice.slack.com/archives/G9YTMSZ2T/p1641064850309500

dictation_mode.talon, add this at the top: 

experiment: anchor-file

modes.talon, add at the bottom: 

^mixed mode$:
  mode.disable("sleep")
  mode.enable("dictation")
  mode.enable("command")

talon deck

Stream deck inspired interactive dashboard for Talon Voice

https://github.com/AndreasArvidsson/talon-deck

push-to-talk

source: https://talonvoice.slack.com/archives/C7ENXA7C4/p1662381706455169?thread_ts=1662379384.654129&cid=C7ENXA7C4 

key(ctrl-m:down):
    speech.enable()
    # app.notify("Talon Wake")
key(ctrl-m:up):
    speech.disable()
    # app.notify("Talon Sleep")

repeat keypress

source: https://talonvoice.slack.com/archives/C7ENXA7C4/p1662565066417169 

key(left left)
key(left:2)

cheatsheets

generate own cheatsheat based on user profile

https://github.com/tararoys/talon-cheatsheet

Mouse Input Alternatives

Name (Costs) Link Hardware Requirements Software Requirements Operating System Tracker Type
CameraMouse (FREE) http://cameramouse.org webcam - (W) Head
CVC Eye Tracker (fork of OpenGazer) (FREE) https://github.com/tiendan/OpenGazer webcam - (A) (L) Eye
Enable Viacam (FREE) https://eviacam.crea-si.com webcam - (L) (W) Head
EyeMine - OptiKey fork optimized to play Minecraft (FREE) https://github.com/SpecialEffect/EyeMine eye tracker (https://github.com/Optikey/Optikey/wiki/Supported-eye-trackers) - (W) Eye
FaceTrackNoIR ($$$) http://www.facetracknoir.nl/home/default.htm webcam / eye tracker - (W) Eye or Head
Head Mouse (DIY) https://www.instructables.com/Head-Mouse-Game-controller-or-disability-aid/ Arduino + - (A) (L) (W) Head
HeadMouse Nano ($$$) https://www.orin.com/access/headmouse/ - - (A) (W) Head
Head Tracking System (DIY) https://www.youtube.com/watch?v=5Nimz4M-sxU PS3 Eye Cam + OpenTrack / Drivers for PS3 Eye Cam (?) Head
KinesicMouse ($$$) http://kinesicmouse.xcessity.at iPhone or iPad - (W) Head
OptiKey (FREE) http://www.optikey.org eye tracker (https://github.com/Optikey/Optikey/wiki/Supported-eye-trackers) - (W) Eye
Physical Presenter Mouse ($$$) https://www.amazon.co.uk/dp/B0B4JVTN4F/ - - (A) (L) (W) Head?
Precision Gaze Mouse (FREE) https://precisiongazemouse.org webcam (and eye tracker) - (W) Eye and Head
Project Gameface (FREE) https://github.com/google/project-gameface webcam - (W) Head and Face
Project IRIS ($$$) http://iris.xcessity.at tobii EyeX, tobii 4C or tobii Eye Tracker 5 Tobii EyeX Engine (W) Eye
Quha Zono (2/X) ($$$) https://www.quha.com - - (A) (W) Head (Gyroscopic)
talon script - dense mouse grid https://github.com/tararoys/dense-mouse-grid microphone - (A) (L) (W) -
talon script - flex mouse grid https://github.com/brollin/flex-mouse-grid microphone - (A) (L) (W) -
talon script - hissing mouse https://github.com/tararoys/hiss-mouse microphone - (A) (L) (W) -
Tracky Mouse (FREE) https://github.com/1j01/tracky-mouse - - (A) (L) (W) Eye
warpd (FREE) https://github.com/rvaiya/warpd - - (A) (L) -
Windows - Ease of Access (FREE) https://support.microsoft.com/en-gb/windows/get-started-with-eye-control-in-windows-1a170a20-1083-2452-8f42-17a7d4fe89a9#bkmk_usewin_win10 Tobii Eye Tracker Tobii Eye Tracking Core Software (W) Eye

Operating System: A: Apple MacOS; L: Linux; W: Windows

Operating System Navigation (like Vim)

Name (Costs) Link Software Requirements Operating System
fluent search (FREE) https://fluentsearch.net/ - (W)
hunt-n-peck (FREE) https://github.com/zsims/hunt-and-peck - (W)
Homerow ($$$) https://www.homerow.app - (A)
invader-vim (FREE) https://github.com/bluecamel/invader_vim python (L)
kindaVim ($$$) https://kindavim.app/ - (A)
Shortcat (FREE) https://shortcat.app - (A)
Vimac (is now Homerow) (FREE) https://vimacapp.com/ - (A)
VimMotion (FREE) https://github.com/dwarvesf/VimMotionApp - (A)
VoiceComputer ($$$) https://voicecomputer.com dragon naturally speaking (W)
win-vind (FREE) https://pit-ray.github.io/win-vind/ - (W)

Operating System: A: Apple MacOS; L: Linux; W: Windows

help window too tall

lower imgui.scale and user.help_max_command_lines_per_page in settings.talon inside knausj_talon folder

windows installation considerations

the cursor doesn't show on login by default

it only appears once you move the mouse - so eyetracking doesn't work until you move a mouse.

To fix this one has to create the registry key: 

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System]
"EnableCursorSuppression"=dword:00000000

eyetracking stops working after an elevation prompt

Lower UAC (User Account Control) one step to: "Notify me only when apps try to make changes to my computer (do not dim my desktop)"

disable sound enhancement

when this option is enabled the pop sound might get interpreted as 'bat', 'od' or 'Paul' or '£'

in the microphone properties (sound settings > recording > properties > advanced > signal enhancements > enable audio enhancements) uncheck the option enable audio enhancements

.talon

os: linux
os: windows
app.name: Meld

the same keys are OR'd by default, different keys are AND'd by default

hardware

microphone

  • Audio Technica BP894
  • Blue Yeti
  • Jabra Evolve 40 UC Professional Wired Headset, Mono
  • Jabra Evolve 75s
  • Samson q9u

linux: function keys f13 - f24 (ERROR invalid hotkey f13)

there is also 'xkb' / 'xkbmap' which I did not look into

if the function keys are not available talon log will show the message 

ERROR invalid hotkey f13

this can be fixed using xmodmap and creating / editing the file ~/.Xmodmap 

keycode 191 = F13 F13 F13
keycode 192 = F14 F14 F14
keycode 193 = F15 F15 F15
keycode 194 = F16 F16 F16
keycode 195 = F17 F17 F17
keycode 196 = F18 F18 F18
keycode 197 = F19 F19 F19
keycode 198 = F20 F20 F20
keycode 199 = F21 F21 F21
keycode 200 = F22 F22 F22
keycode 201 = F23 F23 F23
keycode 202 = F24 F24 F24

to load the file run xmodmap ~/.Xmodmap

to make it available after a reboot 

 echo 'xmodmap ~/.Xmodmap' >> ~/.xinitrc

insert multiline text

^insert multiline text$:
    """
    this is the first line
    and the second line
    line three ...
    """

RUST beta

stream deck support

stream deck software/services has to be stopped

example

# pressing buttons
deck(button_1):
deck(button_2): ... etc
deck(pedal_left):
deck(pedal_middle):
deck(pedal_right):

# for the Stream Deck plus:
# pressing the dial like a button
deck(dial_1):
deck(dial_2): ... etc
# rotating the dial
deck(dial_1:change): print(value)
# touch screen tap
deck(touch):
    print("{x} {y}")

gamepad support

link to test gamepad: https://gamepad-tester.com/

example

gamepad(left_xy:repeat):
    x = x * 20
    y = y * -60
    mouse_scroll(y, x)

gamepad(north): key(cmd-`)
gamepad(south): key(cmd-tab)

gamepad(r1:down): mouse_drag(0)
gamepad(r1:up):   mouse_release(0)

gamepad(l1:down): mouse_drag(1)
gamepad(l1:up):   mouse_release(1)
    
gamepad(right_xy:repeat):
    mx = mouse_x()
    my = mouse_y()
    x = x * 25
    y = y * 25
    mouse_move(mx + x, my - y)

axis

left_xy   left_x   left_y   left_z
right_xy  right_x  right_y  right_z
dpad_xy   dpad_x   dpad_y
north south east west

buttons

c
z
l1 l2 l3
r1 r2 r3
select start mode
dpad_up dpad_down dpad_left dpad_right

additional words vs. words to replace

dictation mode: use words to replace

don't use additional words for replacement, because it also boosts the word priority

additional words example

case: saying root becomes route

additional_words.csv: 

root,route
route,rout

That tells Talon that I want the word root pronounced like how it thinks route sounds, and route when saying rout

macos terminal specific

pf secure keyboard input is enabled, talon can't see hotkeys 

terminal menu: checkbox "secure input"

macos zoom mouse / mouse grid

system preferences -> privacy -> screen recording, need to allow talon

UK vs. US

add replacements to knausj_talon/settings/words_to_replace.csv

sources

rango

inspect toggles

In the background script/service worker enter this in the console:

  • Firefox: await browser.storage.local.get("hintsToggle")
  • Chrome: await chrome.storage.local.get("hintsToggle")

enter background script/service worker

  • In Firefox go to about:debugging#/runtime/this-firefox and click inspect
  • In chrome chrome://extensions, enable developer mode and click on "service worker"

wait

there are insert_wait (for text) and key_wait (for chained commands) which can be configured in a .talon file under settings():

these might be used configured if keys/inputs get mixed up

dictate until keyword "over"

source: https://talonvoice.slack.com/archives/C7ENXA7C4/p1687829353303819?thread_ts=1687827806.922489&cid=C7ENXA7C4 

dictate [<phrase>]$:
    mode.enable("dictation")
    user.parse_phrase(phrase or "")

add to `file core/modes/dictation_mode.talon` 

over [<phrase>]$:
    user.command_mode(phrase or "")

disable voice input when app is in focus

source: https://talonvoice.slack.com/archives/C7ENXA7C4/p1688095980371559?thread_ts=1688077541.629219&cid=C7ENXA7C4 

from talon import actions, ui

def sleep_talon(app):
    if app.name in ["Slack", "Discord", "Teams"]:
        actions.speech.disable()

ui.register('app_activate', sleep_talon)