Release v0.6.10: preserve typing during correction and add personal dictionaries

This commit is contained in:
2026-10-29 22:44:08 +02:00
parent 5c786e12a9
commit 8b5f632ad7
20 changed files with 362 additions and 35 deletions
+1 -1
View File
@@ -1,5 +1,5 @@
cmake_minimum_required(VERSION 3.24) cmake_minimum_required(VERSION 3.24)
project(KSwitcher VERSION 0.6.9 LANGUAGES CXX RC) project(KSwitcher VERSION 0.6.10 LANGUAGES CXX RC)
if(NOT WIN32) if(NOT WIN32)
message(FATAL_ERROR "KSwitcher uses native Windows APIs") message(FATAL_ERROR "KSwitcher uses native Windows APIs")
endif() endif()
+12 -3
View File
@@ -343,14 +343,23 @@ after Space, not immediately after a punctuation key. Save Settings to apply cha
**Frequency rank limit** defaults to **0 (entire dictionary)**. A positive value allows corrections **Frequency rank limit** defaults to **0 (entire dictionary)**. A positive value allows corrections
only to words with a rank at or below that number (for example, 10000). All dictionary words still only to words with a rank at or below that number (for example, 10000). All dictionary words still
protect already valid input, and ambiguous alternatives remain unchanged regardless of this limit. protect already valid input, and ambiguous alternatives remain unchanged regardless of this limit.
**My dictionary** provides separate English, Ukrainian and Russian fields for correct words missing
from the bundled dictionaries. Enter words separated by commas or new lines, then click OK and Save
in Settings. Case and duplicates are normalized automatically; no file editing or ranks are needed.
These entries are stored in settings and survive dictionary updates. They protect correct input and
are eligible as correction targets at any frequency limit. The language and target layout must still
be enabled. **Never correct these words** is a separate exception list and takes precedence.
**Open dictionary folder** opens the files beside the running executable. After editing, click OK **Open dictionary folder** opens the files beside the running executable. After editing, click OK
in the options window and Save in Settings to validate and reload the dictionaries without restarting. in the options window and Save in Settings to validate and reload the dictionaries without restarting.
Use UTF-8, one lowercase word followed by a Tab and a positive rank per line; keep words sorted and Use UTF-8, one lowercase word followed by a Tab and a positive rank per line. Entries can be appended;
unique. For a personal addition, rank 1 makes it eligible at every positive limit. Back up your edited loading sorts them and merges duplicates using the lowest rank. Back up your edited
files before replacing dictionaries with a new distribution. files before replacing dictionaries with a new distribution.
After Space and a 250 ms typing pause, KSwitcher checks the last physically typed word against local After Space and a 250 ms typing pause, KSwitcher checks the last physically typed word against local
dictionaries. For example, `ghbdsn ` can become `привіт `. Continuing to type cancels pending work. dictionaries. For example, `ghbdsn ` can become `привіт `. Continuing to type before selection cancels
pending work. During the replacement itself, key presses and releases are queued and replayed after
clipboard restoration and layout switching, preserving typing order, modifiers and Backspace.
If the active window changes, queued typing is not sent to another window and a warning is shown.
It only corrects unknown words when exactly one alternative is a dictionary word. It only corrects unknown words when exactly one alternative is a dictionary word.
**Minimum word length** defaults to 3 and can be set from 1 to 32 letters. Surrounding punctuation **Minimum word length** defaults to 3 and can be set from 1 to 32 letters. Surrounding punctuation
does not count toward this minimum; the maximum word length is 32 letters. does not count toward this minimum; the maximum word length is 32 letters.
Binary file not shown.
+7 -9
View File
@@ -1,26 +1,24 @@
# Latest Windows release # Latest Windows release
## v0.6.9 — 2026-10-29 ## v0.6.10 — 2026-10-29
## Changes ## Changes
- Show automatic update announcements using the same native Windows notification channel as screenshots. Review update opens the update check and confirmation; Later dismisses the notification. - Preserve typing during automatic layout correction: queue key presses and releases until replacement, clipboard restoration and layout switching finish. Preserve modifiers and Backspace; collapse temporary selection before replay when a correction is rejected.
- Retry notification delivery after five minutes if Windows rejects it, and log update checks and delivery results. Successfully submitted notifications are deduplicated for the current version during the running session. - Add My dictionary fields for English, Ukrainian and Russian in Auto-correction options. Enter correct words separated by commas or new lines; save them with settings without editing dictionary files, sorting entries or assigning ranks.
- Fix Settings flicker when using the mouse wheel: skip layout work at scroll boundaries, move only changed controls in one batch, and avoid erasing the entire window background. - Accept unsorted dictionary files and duplicate entries, retaining the lowest rank for duplicates. Include Ukrainian `реліз` in the bundled vocabulary so `релыз` can be corrected when Russian and Ukrainian are enabled.
## Validation ## Validation
203 native checks, 456 input/layout checks, screenshot editor UI integration, isolated notification registration and activation checks, and 45 updater checks. Regression coverage includes notification retry/deduplication, review-only activation, partial wheel steps, scroll boundaries and batched control movement. Native regression checks cover personal words, settings persistence, language validation, dictionary ordering and duplicate ranks. Input/UI checks cover personal-word entry and normalization. Native editor integration covers typing with Shift and Backspace during successful and rejected corrections. Release checks also cover screenshot UI, isolated notification registration and updater validation.
**Upgrade:** choose Update in the tray menu. Legacy v0.3.0–v0.4.1 clients first receive the v0.4.2 compatibility bridge, then the current full release. v0.2.3 and earlier require manual installation. **Upgrade:** choose Update in the tray menu. Legacy v0.3.0–v0.4.1 clients first receive the v0.4.2 compatibility bridge, then the current full release. v0.2.3 and earlier require manual installation.
For manual installation, exit KSwitcher and extract the entire ZIP into its installation folder. Keep personal `data/` and customized dictionaries. For manual installation, exit KSwitcher and extract the entire ZIP into its installation folder. Keep personal `data/` and customized dictionaries. Existing customized dictionary files may be preserved by the updater; use My dictionary to add missing words without replacing those files.
Download **KSwitcher-win-x64.zip** with **SHA256SUMS.txt**. The package contains all three executables, a release manifest, offline dictionaries, documentation and license notices. It contains no personal settings, clipboard history, Tesseract installation or language models. Download **KSwitcher-win-x64.zip** with **SHA256SUMS.txt**. The package contains all three executables, a release manifest, offline dictionaries, documentation and license notices. It contains no personal settings, clipboard history, Tesseract installation or language models.
Windows notification banners follow system notification and Do not disturb settings. The new update notification behavior takes effect after installing v0.6.9.
Build and package: Build and package:
```powershell ```powershell
.\build.ps1 -Test .\build.ps1 -Stage -Test -Integration
.\package.ps1 -PublishRelease .\package.ps1 -PublishRelease
``` ```
+1 -1
View File
@@ -1 +1 @@
cd6b3dd62065e50b68cf77344fbad8d7be41467142f31527933342f00e3c0f63 KSwitcher-win-x64.zip 36966eb0b2b0af47c245aec6337a16703e0fbdca4d3eda7a8f25fa14a20f347e KSwitcher-win-x64.zip
+12 -2
View File
@@ -17,9 +17,19 @@ replace counts with original frequency ranks, sort lexically. Reproduce with
hashes. The lists contain word forms from subtitles, not a complete spelling hashes. The lists contain word forms from subtitles, not a complete spelling
or grammar dictionary. Recognition can miss names, jargon and uncommon forms. or grammar dictionary. Recognition can miss names, jargon and uncommon forms.
File format: UTF-8 word, TAB, positive original frequency rank, newline. Supplemental Ukrainian vocabulary: `реліз`. These additions use rank 50001,
after the upstream 50k list, rather than claiming a measured corpus frequency.
They are included with the default unlimited rank setting and recorded in sources.json.
File format: UTF-8 word, TAB, positive frequency rank (or supplemental rank), newline.
All entries protect known words. The frequency rank limit in Auto-correction options defaults to 0 All entries protect known words. The frequency rank limit in Auto-correction options defaults to 0
(entire dictionary); a positive limit only permits correction candidates at or below that rank. (entire dictionary); a positive limit only permits correction candidates at or below that rank.
Personal words may use rank 1. Keep entries lowercase, unique and sorted, with a Tab separator. Prefer Settings > Auto-correction options > My dictionary for personal words.
Enter correct spellings in the matching language field, separated by commas or new lines.
These words are stored in settings, survive dictionary updates, and need no ranks or sorting.
Click OK, then Save in Settings to apply them.
For advanced file edits, personal words may use rank 1. Keep entries lowercase with a Tab separator.
Words may be appended: loading sorts entries and merges duplicates using the lowest rank.
Save the UTF-8 file, then click OK in Auto-correction options and Save in Settings to reload it. Save the UTF-8 file, then click OK in Auto-correction options and Save in Settings to reload it.
Back up edited files before replacing dictionaries with a new distribution. Back up edited files before replacing dictionaries with a new distribution.
+6 -3
View File
@@ -9,9 +9,12 @@
"uk": { "uk": {
"source": "https://raw.githubusercontent.com/hermitdave/FrequencyWords/525f9b560de45753a5ea01069454e72e9aa541c6/content/2016/uk/uk_50k.txt", "source": "https://raw.githubusercontent.com/hermitdave/FrequencyWords/525f9b560de45753a5ea01069454e72e9aa541c6/content/2016/uk/uk_50k.txt",
"sourceSha256": "9548b79a608d33f39c5546037a69117d679730b1fda08dd0ce67c043dbf3335a", "sourceSha256": "9548b79a608d33f39c5546037a69117d679730b1fda08dd0ce67c043dbf3335a",
"words": 42549, "words": 42550,
"bytes": 898210, "bytes": 898227,
"sha256": "b4104898dcccbc1331ec01e9cd690e30285fe58481460e47438e973964d09802" "sha256": "c94605e6be676d2d64f89a3bc16f92b9128f9977d226a97b103cf22a3e6835e3",
"supplementalWords": {
"\u0440\u0435\u043b\u0456\u0437": 50001
}
}, },
"ru": { "ru": {
"source": "https://raw.githubusercontent.com/hermitdave/FrequencyWords/525f9b560de45753a5ea01069454e72e9aa541c6/content/2016/ru/ru_50k.txt", "source": "https://raw.githubusercontent.com/hermitdave/FrequencyWords/525f9b560de45753a5ea01069454e72e9aa541c6/content/2016/ru/ru_50k.txt",
+1
View File
@@ -31207,6 +31207,7 @@
релігію 26138 релігію 26138
релігія 22202 релігія 22202
релігії 23427 релігії 23427
реліз 50001
рембо 28603 рембо 28603
ремень 23872 ремень 23872
ремені 28385 ремені 28385
+3 -3
View File
@@ -9,8 +9,8 @@ IDC_SCREENSHOT_PENCIL CURSOR "icons/lucide/pencil.cur"
1 RT_MANIFEST "app.manifest" 1 RT_MANIFEST "app.manifest"
201 RCDATA "update.ps1" 201 RCDATA "update.ps1"
1 VERSIONINFO 1 VERSIONINFO
FILEVERSION 0,6,9,0 FILEVERSION 0,6,10,0
PRODUCTVERSION 0,6,9,0 PRODUCTVERSION 0,6,10,0
FILEOS VOS_NT_WINDOWS32 FILEOS VOS_NT_WINDOWS32
FILETYPE VFT_APP FILETYPE VFT_APP
BEGIN BEGIN
@@ -20,7 +20,7 @@ BEGIN
BEGIN BEGIN
VALUE "FileDescription", "KSwitcher — native keyboard and clipboard utility" VALUE "FileDescription", "KSwitcher — native keyboard and clipboard utility"
VALUE "ProductName", "KSwitcher" VALUE "ProductName", "KSwitcher"
VALUE "FileVersion", "0.6.9" VALUE "FileVersion", "0.6.10"
END END
END END
BLOCK "VarFileInfo" BLOCK "VarFileInfo"
+87 -2
View File
@@ -598,6 +598,27 @@ bool App::handleKeyboard(const KBDLLHOOKSTRUCT &key, bool down, UINT mods)
// down AND up events use exactly the same path as physical keyboard input. // down AND up events use exactly the same path as physical keyboard input.
if (ownKeyboardInput(key) || installingUpdate) if (ownKeyboardInput(key) || installingUpdate)
return false; return false;
{
std::lock_guard lock(stateMutex);
if (typingTarget && GetForegroundWindow() == typingTarget)
{
INPUT input{};
input.type = INPUT_KEYBOARD;
input.ki.wVk = static_cast<WORD>(key.vkCode);
input.ki.wScan = static_cast<WORD>(key.scanCode);
input.ki.dwFlags = (down ? 0 : KEYEVENTF_KEYUP) |
((key.flags & LLKHF_EXTENDED) ? KEYEVENTF_EXTENDEDKEY : 0);
if (key.vkCode == VK_PACKET)
{
input.ki.wVk = 0;
input.ki.dwFlags |= KEYEVENTF_UNICODE;
}
// Replay as ordinary input so tracking and shortcuts see it once,
// after the replacement and layout switch have finished.
deferredTyping.push_back(input);
return true;
}
}
DWORD captureProcess{}; DWORD captureProcess{};
if (hotkeyCapture) GetWindowThreadProcessId(hotkeyCapture, &captureProcess); if (hotkeyCapture) GetWindowThreadProcessId(hotkeyCapture, &captureProcess);
if (hotkeyCapture && (!captureProcess || captureProcess != hotkeyCaptureProcess)) if (hotkeyCapture && (!captureProcess || captureProcess != hotkeyCaptureProcess))
@@ -640,6 +661,41 @@ bool App::handleKeyboard(const KBDLLHOOKSTRUCT &key, bool down, UINT mods)
return true; return true;
return false; return false;
} }
bool App::beginTypingDeferral(HWND target, uint64_t version)
{
std::lock_guard lock(stateMutex);
if (inputVersion != version || GetForegroundWindow() != target || !modifiersReleased())
return false;
typingTarget = target;
return true;
}
void App::replayTyping()
{
std::vector<INPUT> inputs;
HWND target;
{
std::lock_guard lock(stateMutex);
target = typingTarget;
typingTarget = nullptr;
inputs.swap(deferredTyping);
}
busy = false;
if (inputs.empty())
return;
if (GetForegroundWindow() != target)
{
resetInput();
report(L"Typing during correction was not replayed because the active window changed.");
return;
}
// Runs on the hook thread: no physical key can pass between releasing the
// gate and submitting the entire ordered batch to the Windows input queue.
if (SendInput(static_cast<UINT>(inputs.size()), inputs.data(), sizeof(INPUT)) != inputs.size())
{
resetInput();
report(L"Windows could not replay all typing buffered during correction.");
}
}
LRESULT CALLBACK App::keyProc(int code, WPARAM wp, LPARAM lp) LRESULT CALLBACK App::keyProc(int code, WPARAM wp, LPARAM lp)
{ {
if (code == HC_ACTION && current && if (code == HC_ACTION && current &&
@@ -762,11 +818,16 @@ void App::transform(int command, int snippet)
struct BusyReset struct BusyReset
{ {
std::atomic<bool> &b; std::atomic<bool> &b;
HWND window;
bool deferred{};
~BusyReset() ~BusyReset()
{ {
b = false; if (deferred)
PostMessageW(window, ReplayTypingMessage, 0, 0);
else
b = false;
} }
} reset{busy}; } reset{busy, window};
auto start = GetTickCount64(); auto start = GetTickCount64();
std::wstring reason = L"cancelled-before-selection"; std::wstring reason = L"cancelled-before-selection";
std::wstring text, result, expected; std::wstring text, result, expected;
@@ -849,6 +910,21 @@ void App::transform(int command, int snippet)
return; return;
} }
reason = L"clipboard-snapshot"; reason = L"clipboard-snapshot";
if (command == AutoCorrect)
{
if (!beginTypingDeferral(target, version))
return;
reset.deferred = true;
}
// Collapse a temporary selection on every failed auto-correction,
// including clipboard exceptions, before queued typing is replayed.
bool autoSelected = false;
struct SelectionReset
{
bool &selected;
std::function<bool()> valid;
~SelectionReset() { if (selected && valid()) chord(VK_RIGHT); }
} selectionReset{autoSelected, valid};
Transaction tx(*this, target, version); Transaction tx(*this, target, version);
reason = L"selection-unavailable"; reason = L"selection-unavailable";
diagnosticEvent(L"Transform target=" + processName(target) + L" command=" + std::to_wstring(command) + diagnosticEvent(L"Transform target=" + processName(target) + L" command=" + std::to_wstring(command) +
@@ -865,6 +941,7 @@ void App::transform(int command, int snippet)
// conversion, automatic correction must never fall back to a suffix. // conversion, automatic correction must never fall back to a suffix.
if (!tx.valid() || !chord(VK_LEFT, MOD_CONTROL | MOD_SHIFT)) if (!tx.valid() || !chord(VK_LEFT, MOD_CONTROL | MOD_SHIFT))
return; return;
autoSelected = true;
Sleep(25); Sleep(25);
text = tx.copy(); text = tx.copy();
// Some editors first select only ") ". Extend by whole-word // Some editors first select only ") ". Extend by whole-word
@@ -881,7 +958,10 @@ void App::transform(int command, int snippet)
if (!suggestion || !tx.valid() || !autoCorrectionAllowed(target, config)) if (!suggestion || !tx.valid() || !autoCorrectionAllowed(target, config))
{ {
if (tx.valid()) if (tx.valid())
{
chord(VK_RIGHT); chord(VK_RIGHT);
autoSelected = false;
}
return; return;
} }
to = suggestion->target; to = suggestion->target;
@@ -1018,6 +1098,8 @@ void App::transform(int command, int snippet)
sent = tx.paste(textClip(result)); sent = tx.paste(textClip(result));
else if (automatic && tx.valid()) else if (automatic && tx.valid())
chord(VK_RIGHT); chord(VK_RIGHT);
if (sent || result == text)
autoSelected = false;
reason = result == text ? L"conversion-unchanged" : reason = result == text ? L"conversion-unchanged" :
sent ? L"paste-sent-unverified" : L"paste-cancelled"; sent ? L"paste-sent-unverified" : L"paste-cancelled";
if (result == text && if (result == text &&
@@ -1377,6 +1459,9 @@ LRESULT CALLBACK App::windowProc(HWND hwnd, UINT msg, WPARAM wp, LPARAM lp)
case WM_HOTKEY: case WM_HOTKEY:
app->dispatch((int)wp); app->dispatch((int)wp);
return 0; return 0;
case ReplayTypingMessage:
app->replayTyping();
return 0;
case TrayMessage: case TrayMessage:
if (LOWORD(lp) == WM_CONTEXTMENU) if (LOWORD(lp) == WM_CONTEXTMENU)
app->trayMenu(); app->trayMenu();
+5
View File
@@ -16,6 +16,7 @@ inline constexpr UINT TrayMessage = WM_APP + 1, ErrorMessage = WM_APP + 2, Histo
inline constexpr int Toggle = 1, Case = 2, Translit = 3, Undo = 4, ShowHistory = 5, AutoCorrect = 6, inline constexpr int Toggle = 1, Case = 2, Translit = 3, Undo = 4, ShowHistory = 5, AutoCorrect = 6,
Screenshot = 7, Direct = 100; Screenshot = 7, Direct = 100;
inline constexpr UINT AutoCorrectMessage = WM_APP + 6; inline constexpr UINT AutoCorrectMessage = WM_APP + 6;
inline constexpr UINT ReplayTypingMessage = WM_APP + 30;
inline constexpr UINT CaptureHotkeyMessage = WM_APP + 5; inline constexpr UINT CaptureHotkeyMessage = WM_APP + 5;
struct LastTransform struct LastTransform
{ {
@@ -105,6 +106,10 @@ class App
int previousAttemptCommand{}, repeatAttempts{}; int previousAttemptCommand{}, repeatAttempts{};
std::vector<std::pair<int, Hotkey>> hookBindings; std::vector<std::pair<int, Hotkey>> hookBindings;
HotkeyHook hotkeyHook; HotkeyHook hotkeyHook;
HWND typingTarget{};
std::vector<INPUT> deferredTyping;
bool beginTypingDeferral(HWND, uint64_t);
void replayTyping();
static LRESULT CALLBACK windowProc(HWND, UINT, WPARAM, LPARAM); static LRESULT CALLBACK windowProc(HWND, UINT, WPARAM, LPARAM);
static LRESULT CALLBACK keyProc(int, WPARAM, LPARAM); static LRESULT CALLBACK keyProc(int, WPARAM, LPARAM);
static LRESULT CALLBACK mouseProc(int, WPARAM, LPARAM); static LRESULT CALLBACK mouseProc(int, WPARAM, LPARAM);
+21 -5
View File
@@ -70,9 +70,18 @@ AutoCorrector::Dictionary::Dictionary(const std::filesystem::path &path)
} }
if (offsets.empty()) if (offsets.empty())
throw std::runtime_error("Empty auto-correction dictionary"); throw std::runtime_error("Empty auto-correction dictionary");
for (size_t i = 1; i < offsets.size(); ++i) auto entry = [&](uint32_t offset) { return std::string_view(bytes).substr(offset, bytes.find('\t', offset) - offset); };
if (word(i - 1) >= word(i)) auto frequency = [&](uint32_t offset)
throw std::runtime_error("Dictionary must be uniquely sorted"); {
int rank{};
auto start = offset + entry(offset).size() + 1;
std::from_chars(bytes.data() + start, bytes.data() + bytes.size(), rank);
return rank;
};
std::sort(offsets.begin(), offsets.end(), [&](uint32_t a, uint32_t b)
{ return entry(a) == entry(b) ? frequency(a) < frequency(b) : entry(a) < entry(b); });
offsets.erase(std::unique(offsets.begin(), offsets.end(), [&](uint32_t a, uint32_t b)
{ return entry(a) == entry(b); }), offsets.end());
offsets.shrink_to_fit(); offsets.shrink_to_fit();
bytes.shrink_to_fit(); bytes.shrink_to_fit();
} }
@@ -151,9 +160,16 @@ std::optional<AutoSuggestion> AutoCorrector::suggest(const std::wstring &typed,
auto sourceLanguage = layoutLanguage(layouts[source]); auto sourceLanguage = layoutLanguage(layouts[source]);
if (!dictionaries.contains(sourceLanguage)) if (!dictionaries.contains(sourceLanguage))
return {}; return {};
auto personal = [&](const std::wstring &language, const std::wstring &candidate)
{
auto found = settings.autoWords.find(language);
return found != settings.autoWords.end() &&
std::any_of(found->second.begin(), found->second.end(), [&](const std::wstring &known)
{ return lower(known) == lower(candidate); });
};
// Valid words from either selected language are never changed. // Valid words from either selected language are never changed.
for (auto &[language, d] : dictionaries) for (auto &[language, d] : dictionaries)
if (d.rank(word)) if (d.rank(word) || personal(language, word))
return {}; return {};
std::optional<AutoSuggestion> candidate; std::optional<AutoSuggestion> candidate;
int matches = 0; int matches = 0;
@@ -168,7 +184,7 @@ std::optional<AutoSuggestion> AutoCorrector::suggest(const std::wstring &typed,
auto converted = convert(word, layouts[source], layouts[i], true); auto converted = convert(word, layouts[source], layouts[i], true);
if (converted == word || !safeAutoWord(converted, settings.autoMinLength)) if (converted == word || !safeAutoWord(converted, settings.autoMinLength))
continue; continue;
int rank = dictionary->second.rank(converted); int rank = personal(dictionary->first, converted) ? 1 : dictionary->second.rank(converted);
if (!rank) if (!rank)
continue; continue;
if (++matches > 1) if (++matches > 1)
+2
View File
@@ -1,6 +1,7 @@
#pragma once #pragma once
#include <cstdint> #include <cstdint>
#include <functional> #include <functional>
#include <map>
#include <optional> #include <optional>
#include <string> #include <string>
#include <vector> #include <vector>
@@ -58,6 +59,7 @@ struct Settings
std::wstring autoSuffixChars = L")]}»”’\".,!?:;"; std::wstring autoSuffixChars = L")]}»”’\".,!?:;";
std::vector<std::wstring> autoLanguages{L"en", L"uk"}; std::vector<std::wstring> autoLanguages{L"en", L"uk"};
std::vector<std::wstring> autoNever; std::vector<std::wstring> autoNever;
std::map<std::wstring, std::vector<std::wstring>> autoWords;
std::vector<std::wstring> autoExcluded{ std::vector<std::wstring> autoExcluded{
L"Code", L"rider64", L"devenv", L"WindowsTerminal", L"cmd", L"powershell", L"pwsh", L"Code", L"rider64", L"devenv", L"WindowsTerminal", L"cmd", L"powershell", L"pwsh",
L"Far", L"ConEmu64", L"putty", L"KeePass", L"KeePassXC", L"1Password", L"Bitwarden"}; L"Far", L"ConEmu64", L"putty", L"KeePass", L"KeePassXC", L"1Password", L"Bitwarden"};
+22
View File
@@ -92,6 +92,17 @@ void validateSettings(const Settings &s)
throw std::runtime_error("Unsupported auto-correction language"); throw std::runtime_error("Unsupported auto-correction language");
if (s.autoNever.size() > 10000 || s.autoExcluded.size() > 1000) if (s.autoNever.size() > 10000 || s.autoExcluded.size() > 1000)
throw std::runtime_error("Too many auto-correction exceptions"); throw std::runtime_error("Too many auto-correction exceptions");
for (const auto &[language, words] : s.autoWords)
{
std::wstring alphabet = language == L"en" ? L"abcdefghijklmnopqrstuvwxyz" :
language == L"uk" ? L"абвгґдеєжзиіїйклмнопрстуфхцчшщьюя" :
language == L"ru" ? L"абвгдеёжзийклмнопрстуфхцчшщъыьэюя" : L"";
if (alphabet.empty() || words.size() > 10000)
throw std::runtime_error("Invalid personal dictionary language or too many words");
for (const auto &word : words)
if (word.empty() || word.size() > 32 || lower(word).find_first_not_of(alphabet) != word.npos)
throw std::runtime_error("Personal words must contain 1-32 letters of their selected language");
}
if (s.maxItems < 10 || s.maxItems > 10000 || s.maxMiB < 1 || s.maxMiB > 2048 || s.maxImageMiB < 1 || if (s.maxItems < 10 || s.maxItems > 10000 || s.maxMiB < 1 || s.maxMiB > 2048 || s.maxImageMiB < 1 ||
s.maxImageMiB > 64) s.maxImageMiB > 64)
throw std::runtime_error("History limits: 10–10000 items, 1–2048 MiB total, 1–64 MiB per image"); throw std::runtime_error("History limits: 10–10000 items, 1–2048 MiB total, 1–64 MiB per image");
@@ -143,6 +154,10 @@ Settings decodeSettings(const Bytes &bytes, const std::vector<Layout> &layouts)
s.autoMinLength = j.value("autoCorrectMinLength", 3); s.autoMinLength = j.value("autoCorrectMinLength", 3);
s.autoPrefixChars = str(j, "autoCorrectPrefixChars", s.autoPrefixChars); s.autoPrefixChars = str(j, "autoCorrectPrefixChars", s.autoPrefixChars);
s.autoSuffixChars = str(j, "autoCorrectSuffixChars", s.autoSuffixChars); s.autoSuffixChars = str(j, "autoCorrectSuffixChars", s.autoSuffixChars);
if (j.contains("autoCorrectWords"))
for (auto &[language, words] : j.at("autoCorrectWords").items())
for (auto &word : words)
s.autoWords[wide(language)].push_back(lower(wide(word.get<std::string>())));
for (auto pair : {std::pair{"autoCorrectLanguages", &s.autoLanguages}, for (auto pair : {std::pair{"autoCorrectLanguages", &s.autoLanguages},
{"autoCorrectNever", &s.autoNever}, {"autoCorrectNever", &s.autoNever},
{"autoCorrectExcludedApps", &s.autoExcluded}}) {"autoCorrectExcludedApps", &s.autoExcluded}})
@@ -229,6 +244,13 @@ Bytes encodeSettings(const Settings &s)
j["autoCorrectMinLength"] = s.autoMinLength; j["autoCorrectMinLength"] = s.autoMinLength;
j["autoCorrectPrefixChars"] = utf8(s.autoPrefixChars); j["autoCorrectPrefixChars"] = utf8(s.autoPrefixChars);
j["autoCorrectSuffixChars"] = utf8(s.autoSuffixChars); j["autoCorrectSuffixChars"] = utf8(s.autoSuffixChars);
j["autoCorrectWords"] = Json::object();
for (const auto &[language, words] : s.autoWords)
{
auto &items = j["autoCorrectWords"][utf8(language)] = Json::array();
for (const auto &word : words)
items.push_back(utf8(lower(word)));
}
for (auto pair : {std::pair{"autoCorrectLanguages", &s.autoLanguages}, for (auto pair : {std::pair{"autoCorrectLanguages", &s.autoLanguages},
{"autoCorrectNever", &s.autoNever}, {"autoCorrectNever", &s.autoNever},
{"autoCorrectExcludedApps", &s.autoExcluded}}) {"autoCorrectExcludedApps", &s.autoExcluded}})
+22 -5
View File
@@ -1009,6 +1009,15 @@ LRESULT CALLBACK autoOptionsProc(HWND w, UINT msg, WPARAM wp, LPARAM lp)
if (checked(w, 403)) if (checked(w, 403))
draft.autoLanguages.push_back(L"ru"); draft.autoLanguages.push_back(L"ru");
draft.autoNever = parseCommaList(value(w, 405)); draft.autoNever = parseCommaList(value(w, 405));
for (auto [id, language] : {std::pair{411, L"en"}, {412, L"uk"}, {413, L"ru"}})
{
auto text = lower(value(w, id));
std::replace(text.begin(), text.end(), L'\n', L',');
auto words = parseCommaList(text);
std::sort(words.begin(), words.end());
words.erase(std::unique(words.begin(), words.end()), words.end());
draft.autoWords[language] = std::move(words);
}
draft.autoExcluded = parseCommaList(value(w, 404)); draft.autoExcluded = parseCommaList(value(w, 404));
draft.autoPrefixChars = value(w, 408); draft.autoPrefixChars = value(w, 408);
draft.autoSuffixChars = value(w, 409); draft.autoSuffixChars = value(w, 409);
@@ -1017,6 +1026,7 @@ LRESULT CALLBACK autoOptionsProc(HWND w, UINT msg, WPARAM wp, LPARAM lp)
validateSettings(draft); validateSettings(draft);
view->settings->draft.autoLanguages = draft.autoLanguages; view->settings->draft.autoLanguages = draft.autoLanguages;
view->settings->draft.autoNever = draft.autoNever; view->settings->draft.autoNever = draft.autoNever;
view->settings->draft.autoWords = draft.autoWords;
view->settings->draft.autoExcluded = draft.autoExcluded; view->settings->draft.autoExcluded = draft.autoExcluded;
view->settings->draft.autoMaxRank = draft.autoMaxRank; view->settings->draft.autoMaxRank = draft.autoMaxRank;
view->settings->draft.autoMinLength = draft.autoMinLength; view->settings->draft.autoMinLength = draft.autoMinLength;
@@ -1063,13 +1073,20 @@ void showAutoOptions(HWND parent, SettingsView *v)
for (auto &lang : v->draft.autoLanguages) for (auto &lang : v->draft.autoLanguages)
check(w, lang == L"en" ? 401 : lang == L"uk" ? 402 : 403, true); check(w, lang == L"en" ? 401 : lang == L"uk" ? 402 : 403, true);
label(w, L"Excluded applications (process names, comma-separated)", 18, 94, 525); label(w, L"Excluded applications (process names, comma-separated)", 18, 94, 525);
edit(w, 404, 18, 122, 524, 84, ES_MULTILINE | WS_VSCROLL | ES_AUTOVSCROLL); edit(w, 404, 18, 122, 524, 52, ES_MULTILINE | WS_VSCROLL | ES_AUTOVSCROLL);
value(w, 404, commaList(v->draft.autoExcluded)); value(w, 404, commaList(v->draft.autoExcluded));
label(w, L"Personal dictionary: never correct these words (comma-separated)", 18, 220, 525); label(w, L"Never correct these words (comma-separated)", 18, 184, 525);
edit(w, 405, 18, 248, 524, 84, ES_MULTILINE | WS_VSCROLL | ES_AUTOVSCROLL); edit(w, 405, 18, 212, 524, 52, ES_MULTILINE | WS_VSCROLL | ES_AUTOVSCROLL);
value(w, 405, commaList(v->draft.autoNever)); value(w, 405, commaList(v->draft.autoNever));
label(w, L"Corrects after Space and a short pause.", 18, 344, 525); label(w, L"My dictionary: correct words, separated by commas or new lines", 18, 276, 525);
label(w, L"Undo skips this word until KSwitcher restarts.", 18, 366, 525); for (auto [id, language] : {std::pair{411, L"en"}, {412, L"uk"}, {413, L"ru"}})
{
int x = 18 + (id - 411) * 178;
label(w, id == 411 ? L"English" : id == 412 ? L"Ukrainian" : L"Russian", x, 302, 168);
edit(w, id, x, 328, 168, 56, ES_MULTILINE | WS_VSCROLL | ES_AUTOVSCROLL | ES_WANTRETURN);
auto words = v->draft.autoWords.find(language);
if (words != v->draft.autoWords.end()) value(w, id, commaList(words->second));
}
label(w, L"Frequency rank limit (0 = entire dictionary)", 18, 400, 390); label(w, L"Frequency rank limit (0 = entire dictionary)", 18, 400, 390);
edit(w, 406, 420, 396, 122, 26, ES_NUMBER); edit(w, 406, 420, 396, 122, 26, ES_NUMBER);
value(w, 406, std::to_wstring(v->draft.autoMaxRank)); value(w, 406, std::to_wstring(v->draft.autoMaxRank));
+1 -1
View File
@@ -2,7 +2,7 @@
#include "platform.h" #include "platform.h"
namespace ks namespace ks
{ {
inline constexpr wchar_t AppVersion[] = L"0.6.9"; inline constexpr wchar_t AppVersion[] = L"0.6.10";
inline constexpr UINT UpdateMessage = WM_APP + 7; inline constexpr UINT UpdateMessage = WM_APP + 7;
inline constexpr UINT UpdateCheckIntervalMs = 6 * 60 * 60 * 1000; inline constexpr UINT UpdateCheckIntervalMs = 6 * 60 * 60 * 1000;
inline constexpr UINT UpdateNotificationRetryMs = 5 * 60 * 1000; inline constexpr UINT UpdateNotificationRetryMs = 5 * 60 * 1000;
+23
View File
@@ -586,6 +586,29 @@ int main(int argc, char **argv) {
fitted=ks::fitWindowRect({2000,2000,2600,2400},{-1920,0,0,1080}); fitted=ks::fitWindowRect({2000,2000,2600,2400},{-1920,0,0,1080});
require(fitted.left==-600&&fitted.top==680&&fitted.right==0,"negative-coordinate monitor supported"); require(fitted.left==-600&&fitted.top==680&&fitted.right==0,"negative-coordinate monitor supported");
ks::showSettings(app,false); ks::showSettings(app,false);
SendMessageW(app.settingsWindow,WM_COMMAND,255,0);
auto autoOptions=FindWindowW(L"KSwitcher.AutoOptions",nullptr);
require(autoOptions!=nullptr,"auto-correction options open");
ShowWindow(autoOptions,SW_HIDE);
for(int id:{411,412,413}) {
RECT control{},client{};
GetWindowRect(GetDlgItem(autoOptions,id),&control);
MapWindowPoints(nullptr,autoOptions,reinterpret_cast<POINT*>(&control),2);
GetClientRect(autoOptions,&client);
require(control.left>=0&&control.top>=0&&control.right<=client.right&&control.bottom<=client.bottom,
"personal dictionary field fits options window");
}
SetDlgItemTextW(autoOptions,412,L"Релізик\r\nрелізик, словечко");
SendMessageW(autoOptions,WM_COMMAND,IDOK,0);
require(!IsWindow(autoOptions),"personal words accepted without file editing");
SendMessageW(app.settingsWindow,WM_COMMAND,255,0);
autoOptions=FindWindowW(L"KSwitcher.AutoOptions",nullptr);
ShowWindow(autoOptions,SW_HIDE);
wchar_t personalWords[128]{};
GetDlgItemTextW(autoOptions,412,personalWords,128);
require(std::wstring(personalWords)==L"релізик, словечко",
"personal dictionary normalizes case, newlines and duplicate words");
SendMessageW(autoOptions,WM_COMMAND,IDCANCEL,0);
if (!app.history) app.history=std::make_unique<ks::History>(temp/L"history-layout"); if (!app.history) app.history=std::make_unique<ks::History>(temp/L"history-layout");
ks::showHistory(app,false); ks::showHistory(app,false);
auto history=app.historyWindow; auto history=app.historyWindow;
+75
View File
@@ -3,6 +3,17 @@
#include <fstream> #include <fstream>
#include <commctrl.h> #include <commctrl.h>
namespace ks
{
struct InputRegressionAccess
{
static bool deferring(App &app)
{
std::lock_guard lock(app.stateMutex);
return app.typingTarget != nullptr;
}
};
}
namespace namespace
{ {
ks::App *app{}; ks::App *app{};
@@ -11,6 +22,7 @@ ks::Clip saved;
bool captured = false, failed = false; bool captured = false, failed = false;
int stage = 0, ru = -1, en = -1; int stage = 0, ru = -1, en = -1;
size_t punctuationCase = 0; size_t punctuationCase = 0;
bool typeDuringCorrection = false, typedDuringCorrection = false;
const std::vector<std::pair<std::wstring, std::wstring>> punctuationCases{ const std::vector<std::pair<std::wstring, std::wstring>> punctuationCases{
{L"(ljvf) ", L"(дома) "}, {L"[ljvf] ", L"[дома] "}, {L"{ljvf} ", L"{дома} "}, {L"(ljvf) ", L"(дома) "}, {L"[ljvf] ", L"[дома] "}, {L"{ljvf} ", L"{дома} "},
{L"ljvf: ", L"дома: "}, {L"ljvf; ", L"дома; "}, {L"#ljvf%; ", L"#дома%; "}}; {L"ljvf: ", L"дома: "}, {L"ljvf; ", L"дома; "}, {L"#ljvf%; ", L"#дома%; "}};
@@ -25,7 +37,14 @@ size_t diagnosticStart{};
void require(bool ok, const char *name) void require(bool ok, const char *name)
{ {
if (!ok) if (!ok)
{
wchar_t observed[2048]{};
GetWindowTextW(field, observed, 2048);
std::cerr << "Editor at failure: " << ks::utf8(observed)
<< "; foregroundSame=" << (GetForegroundWindow() == host)
<< "; inputVersion=" << app->inputVersion.load() << '\n';
throw std::runtime_error(name); throw std::runtime_error(name);
}
std::cout << "PASS " << name << '\n'; std::cout << "PASS " << name << '\n';
} }
std::wstring text() std::wstring text()
@@ -80,7 +99,26 @@ LRESULT CALLBACK procedure(HWND w, UINT msg, WPARAM wp, LPARAM lp)
if (GetTickCount64() > deadline) if (GetTickCount64() > deadline)
throw std::runtime_error("Native integration timed out"); throw std::runtime_error("Native integration timed out");
if (app->busy) if (app->busy)
{
if (typeDuringCorrection && ks::InputRegressionAccess::deferring(*app))
{
typeDuringCorrection = false;
typedDuringCorrection = true;
std::vector<INPUT> inputs;
for (auto [key, up] : std::vector<std::pair<WORD, bool>>{
{'F', false}, {'F', true}, {VK_LSHIFT, false},
{'F', false}, {'F', true}, {VK_LSHIFT, true},
{'B', false}, {'B', true}, {VK_BACK, false}, {VK_BACK, true}})
{
auto input = ks::keyboardInput(key, up ? KEYEVENTF_KEYUP : 0);
input.ki.dwExtraInfo = 123; // External input must traverse the actual hook.
inputs.push_back(input);
}
require(SendInput((UINT)inputs.size(), inputs.data(), sizeof(INPUT)) == inputs.size(),
"concurrent typing submitted during selection/paste transaction");
}
return 0; return 0;
}
switch (stage++) switch (stage++)
{ {
case 0: case 0:
@@ -242,6 +280,41 @@ LRESULT CALLBACK procedure(HWND w, UINT msg, WPARAM wp, LPARAM lp)
break; break;
case 17: case 17:
require(text() == punctuationCases.back().first, "undo preserves custom punctuation"); require(text() == punctuationCases.back().first, "undo preserves custom punctuation");
[[fallthrough]];
case 20:
if (!captured)
{
saved = ks::readClipboard(app->window, 64 * 1024 * 1024);
captured = true;
app->ownSequence = ks::writeClipboard(app->window, ks::textClip(sentinel));
}
app->settings.autoCorrect = true;
app->settings.switchLayout = true;
app->settings.autoLanguages = {L"en", L"ru"};
app->settings.cycle = {app->layouts[en].id, app->layouts[ru].id};
app->autoCorrector = ks::AutoCorrector{};
ActivateKeyboardLayout(app->layouts[en].handle, 0);
prepare(L"ghbdtn ", false);
record(L"ghbdtn ");
typeDuringCorrection = true;
app->transform(ks::AutoCorrect);
stage = 18;
break;
case 18:
require(typedDuringCorrection && text() == L"привет аА",
"typing during correction replays in order with shift and backspace in new layout");
require(ks::readClipboard(app->window).text() == sentinel,
"concurrent typing preserves clipboard restoration");
ActivateKeyboardLayout(app->layouts[en].handle, 0);
prepare(L"OLDghbdtn ", false);
record(L"ghbdtn ");
typedDuringCorrection = false;
typeDuringCorrection = true;
app->transform(ks::AutoCorrect);
break;
case 19:
require(typedDuringCorrection && text() == L"OLDghbdtn fF",
"rejected correction collapses selection before replaying typing in original layout");
finish(); finish();
break; break;
} }
@@ -366,6 +439,8 @@ int main(int argc, char **argv)
CreateWindowW(L"EDIT", L"", WS_CHILD | WS_VISIBLE | WS_BORDER | ES_MULTILINE | ES_WANTRETURN, CreateWindowW(L"EDIT", L"", WS_CHILD | WS_VISIBLE | WS_BORDER | ES_MULTILINE | ES_WANTRETURN,
20, 20, 590, 230, host, nullptr, cls.hInstance, nullptr); 20, 20, 590, 230, host, nullptr, cls.hInstance, nullptr);
deadline = GetTickCount64() + 30000; deadline = GetTickCount64() + 30000;
if (argc > 1 && std::string(argv[1]) == "--typing-only")
stage = 20;
SetTimer(host, 1, 100, nullptr); SetTimer(host, 1, 100, nullptr);
app->execute(); app->execute();
} }
+55
View File
@@ -241,6 +241,37 @@ int main(int argc, char **argv)
require(corrector.memoryBytes() < 3 * 1024 * 1024, "two dictionaries fit within 3 MiB"); require(corrector.memoryBytes() < 3 * 1024 * 1024, "two dictionaries fit within 3 MiB");
require(!ks::autoCorrectionAllowed(nullptr, autoSettings), "missing focused editor fails closed"); require(!ks::autoCorrectionAllowed(nullptr, autoSettings), "missing focused editor fails closed");
ks::AutoCorrector missing; ks::AutoCorrector missing;
{
ks::AutoCorrector supplemental;
auto options = autoSettings;
options.autoLanguages = {L"en", L"uk", L"ru"};
options.cycle = {L"en", L"uk", L"ru"};
supplemental.configure(L"dictionaries", options.autoLanguages);
auto release = supplemental.suggest(L"релыз ", 1, layouts, options);
require(release && release->text == L"реліз " && release->target == 2,
"supplemental Ukrainian release word corrects from Russian layout");
require(!supplemental.suggest(L"реліз ", 2, layouts, options),
"correct Ukrainian release word stays unchanged");
require(!supplemental.suggest(L"релиз ", 1, layouts, options),
"valid Russian release word stays unchanged");
options.autoMaxRank = 50000;
require(!supplemental.suggest(L"релыз ", 1, layouts, options),
"supplemental vocabulary respects explicit frequency limit");
options.autoWords[L"uk"] = {L"релізик"};
options.autoMaxRank = 1;
auto custom = supplemental.suggest(L"релызик ", 1, layouts, options);
require(custom && custom->text == L"релізик " && custom->target == 2,
"personal word enables correction without editing dictionary files");
require(!supplemental.suggest(L"релізик ", 2, layouts, options),
"personal word is protected from correction");
options.autoNever = {L"релызик"};
require(!supplemental.suggest(L"релызик ", 1, layouts, options),
"never-correct list takes precedence over personal word");
options.autoNever.clear();
options.cycle = {L"en", L"ru"};
require(!supplemental.suggest(L"релызик ", 1, layouts, options),
"personal dictionary respects disabled target layouts");
}
rejects([&] { missing.configure(L"missing-dictionaries", {L"en"}); }, "missing dictionary rejected"); rejects([&] { missing.configure(L"missing-dictionaries", {L"en"}); }, "missing dictionary rejected");
require(ks::convertTo(L"ghbdtn", layouts, 1, 0) == L"привет", "hint conversion"); require(ks::convertTo(L"ghbdtn", layouts, 1, 0) == L"привет", "hint conversion");
require(ks::convertTo(L"привет", layouts, 1, 1) == L"привет", "same source is unchanged"); require(ks::convertTo(L"привет", layouts, 1, 1) == L"привет", "same source is unchanged");
@@ -335,6 +366,25 @@ int main(int argc, char **argv)
auto temp = std::filesystem::temp_directory_path() / auto temp = std::filesystem::temp_directory_path() /
(L"KSwitcher-tests-" + std::to_wstring(GetCurrentProcessId())); (L"KSwitcher-tests-" + std::to_wstring(GetCurrentProcessId()));
std::filesystem::create_directories(temp); std::filesystem::create_directories(temp);
{
auto folder = temp / L"dictionaries";
std::filesystem::create_directories(folder);
auto put = [&](const wchar_t *name, const std::wstring &text)
{
auto data = ks::utf8(text);
ks::atomicWrite(folder / name, ks::Bytes(data.begin(), data.end()));
};
put(L"en.txt", L"zoo\t3\nhello\t99\nhello\t1\n");
put(L"uk.txt", L"дім\t3\n");
ks::AutoCorrector edited;
edited.configure(folder, {L"en", L"uk"});
auto options = ks::Settings{};
options.cycle = {L"en", L"uk"};
options.autoMaxRank = 1;
auto correction = edited.suggest(L"руддщ ", 2, layouts, options);
require(correction && correction->text == L"hello ",
"unsorted dictionaries and duplicates load using lowest duplicate rank");
}
ks::Pixels pixels{2, 1, {10, 20, 30, 255, 40, 50, 60, 255}}; ks::Pixels pixels{2, 1, {10, 20, 30, 255, 40, 50, 60, 255}};
ks::Clip firstImage{{ks::imageDib(pixels), {0, L"Ole Private Data", {1, 2, 3}}}}; ks::Clip firstImage{{ks::imageDib(pixels), {0, L"Ole Private Data", {1, 2, 3}}}};
auto thumbnail = ks::decodeImage(firstImage, 1, 1); auto thumbnail = ks::decodeImage(firstImage, 1, 1);
@@ -384,11 +434,16 @@ int main(int argc, char **argv)
settings.autoPrefixChars = L"(#"; settings.autoPrefixChars = L"(#";
settings.autoSuffixChars = L");%"; settings.autoSuffixChars = L");%";
settings.autoNever = {L"myWord"}; settings.autoNever = {L"myWord"};
settings.autoWords = {{L"uk", {L"релізик"}}, {L"en", {L"kswitcher"}}};
settings.autoExcluded = {L"Code.exe"}; settings.autoExcluded = {L"Code.exe"};
settings.cycle = {L"en", L"uk"}; settings.cycle = {L"en", L"uk"};
settings.snippets = {{L";address", L"Line 1\r\n{cursor}Line 2"}}; settings.snippets = {{L";address", L"Line 1\r\n{cursor}Line 2"}};
ks::saveSettings(temp / L"settings.json", settings); ks::saveSettings(temp / L"settings.json", settings);
auto restored = ks::loadSettings(temp / L"settings.json", {}); auto restored = ks::loadSettings(temp / L"settings.json", {});
require(restored.autoWords == settings.autoWords, "personal dictionaries survive settings roundtrip");
auto invalidWords = settings;
invalidWords.autoWords[L"uk"] = {L"hello"};
rejects([&] { ks::validateSettings(invalidWords); }, "personal dictionary rejects wrong language letters");
require(ks::encodeSettings(ks::decodeSettings(ks::encodeSettings(settings), {})) == ks::encodeSettings(settings), require(ks::encodeSettings(ks::decodeSettings(ks::encodeSettings(settings), {})) == ks::encodeSettings(settings),
"configurator snapshot survives serialization without changing the concurrency token"); "configurator snapshot survives serialization without changing the concurrency token");
rejects([&] { ks::decodeSettings(ks::Bytes{'{', '}'}, {}); }, rejects([&] { ks::decodeSettings(ks::Bytes{'{', '}'}, {}); },
+6
View File
@@ -3,6 +3,8 @@ from pathlib import Path
import urllib.request, hashlib, json import urllib.request, hashlib, json
ROOT=Path(__file__).resolve().parents[1]/'dictionaries' ROOT=Path(__file__).resolve().parents[1]/'dictionaries'
REV='525f9b560de45753a5ea01069454e72e9aa541c6' REV='525f9b560de45753a5ea01069454e72e9aa541c6'
# Supplemental vocabulary has no corpus frequency: rank after the 50k source list.
SUPPLEMENTAL={'uk': {'реліз': 50001}}
ROOT.mkdir(exist_ok=True) ROOT.mkdir(exist_ok=True)
manifest={} manifest={}
for lang in ('en','uk','ru'): for lang in ('en','uk','ru'):
@@ -15,8 +17,12 @@ for lang in ('en','uk','ru'):
if not (2<=len(word)<=40 and all(c in alphabet for c in word)):continue if not (2<=len(word)<=40 and all(c in alphabet for c in word)):continue
if int(count)<5:continue if int(count)<5:continue
rows.setdefault(word,rank) rows.setdefault(word,rank)
for word,rank in SUPPLEMENTAL.get(lang,{}).items():
rows.setdefault(word,rank)
out=''.join(f'{word}\t{rows[word]}\n' for word in sorted(rows)) out=''.join(f'{word}\t{rows[word]}\n' for word in sorted(rows))
(ROOT/f'{lang}.txt').write_bytes(out.encode('utf-8')) (ROOT/f'{lang}.txt').write_bytes(out.encode('utf-8'))
manifest[lang]={'source':url,'sourceSha256':hashlib.sha256(data).hexdigest(),'words':len(rows),'bytes':len(out.encode()),'sha256':hashlib.sha256(out.encode()).hexdigest()} manifest[lang]={'source':url,'sourceSha256':hashlib.sha256(data).hexdigest(),'words':len(rows),'bytes':len(out.encode()),'sha256':hashlib.sha256(out.encode()).hexdigest()}
if lang in SUPPLEMENTAL:
manifest[lang]['supplementalWords']=SUPPLEMENTAL[lang]
(ROOT/'sources.json').write_text(json.dumps(manifest,indent=2)+'\n',encoding='utf-8') (ROOT/'sources.json').write_text(json.dumps(manifest,indent=2)+'\n',encoding='utf-8')
print(json.dumps(manifest,indent=2)) print(json.dumps(manifest,indent=2))