Commit Graph

231 Commits

Author SHA1 Message Date
toasted-nutbread
a8ff38eec3
Fix display issues (#984)
* Remove unused function

* Update expresionMulti detection

* Simplify kanji early escape

* Simplify frequency/pitch data creation

* Update implementation of _buildTermMeta

* Update how pitch accents sources are collected

* Remove old _buildTermMeta
2020-11-01 21:24:35 -05:00
toasted-nutbread
cee64f91eb
Fix data inconsistencies (#891)
* Simplify variable name

* Add dictionaryNames array

* Update how the dictionary field is assigned

* Use "dictionary" instead of "title" for consistency

* Remove redundant dictionary name

* Use _expandTags to create pitch accent tags
2020-10-06 20:28:49 -04:00
toasted-nutbread
69f94f8cb8
Update merged definitions to have consistent structure (#890)
* Update comment about definitions type

* Add conversion to ensure consistent data structure
2020-10-06 19:31:53 -04:00
toasted-nutbread
7ce87c51b4
Don't add duplicate definitions (#889) 2020-10-05 22:50:10 -04:00
toasted-nutbread
0cfaa1a5a8
Dictionary definition structure fixes (#888)
* Restore support for tags without metadata

* Fix missing spread
2020-10-05 21:10:52 -04:00
toasted-nutbread
8c8c27001e
Fix incorrect definitions array being used for compatibility merged definitions (#887) 2020-10-04 22:21:16 -04:00
toasted-nutbread
54810510fa
Fix primary audio source (#886)
* Add abstraction: _getDefinitionPrimaryExpressionAndReading

* Reuse existing definitions in a sequence

* Revert change to related definition source/rawSource/sourceTerm

* Update _getDefinitionPrimaryExpressionAndReading to return best match
2020-10-04 22:04:44 -04:00
toasted-nutbread
2d3cf89b49
Fix doc comments for translation options (#885) 2020-10-04 19:36:21 -04:00
toasted-nutbread
c604a406f0
Translator organization (#884)
* Group reduction functions together

* Group simple helpers

* Add label comments

* Move core _findTerms functions to top

* Move _getSecondarySearchDictionaryMap and _getDictionaryPriority

* Move tag helpers

* Move _getTextOptionEntryVariants

* Move functions up

* Move functions up

* Update label comments
2020-10-04 19:33:22 -04:00
toasted-nutbread
8b8806484e
Add check to verify expressions are strings (#883) 2020-10-04 19:04:21 -04:00
toasted-nutbread
f904b3e11a
Add sourceTermExactMatchCount to prioritize exact expression matches (#882) 2020-10-04 18:54:03 -04:00
toasted-nutbread
561e36e88d
Translator definition source term (and other info) (#881)
* Add sourceTerm field to 'term' and 'termGrouped' definitions

* Fix comparison of expressions which are always the same

* Rename/restructure term info map

* Add source term

* Add sourceTerm to expressions array

* Reuse furiganaSegments

* Add helper function _createExpressionDetailsListFromTermInfoMap

* Add expressions array to termMergedByGlossary

* Add expressions to definition types 'term' and 'termGrouped'

* Rename expressionDetails* to termDetails*

* Correct the source/rawSource/sourceTerm for related sequenced definitions

* Simplify structure of sequencedDefinitions

* Remove TODO
2020-10-04 18:10:10 -04:00
toasted-nutbread
7e31dcca39
Move deinflection reasons setup to backend (#880)
* Move deinflection reasons setup to backend

* Remove async
2020-10-04 13:09:04 -04:00
toasted-nutbread
2bd82353e4
Translator options refactor (#879)
* Refactor internal options for findTerms to not use the settings object

* Move findTerms/findKanji options creation

* Deconstruct used options values to variables before any await calls

* Rename findTermsOptions to just options

* Add documentation comments

* Add type information about definitions
2020-10-04 12:54:55 -04:00
toasted-nutbread
86c64ac4c2
Translator refactoring (#878)
* Add helper function _getSecondarySearchDictionaryMap

* Use spreads instead of concat

* Create helper function for assigning disambiguation info

* Add _getTermTagsScoreSum helper

* Rename variables

* Move assignment of definition details

* Convert databaseDefinitions to standard definitions

* Move creation of definitions

* Always pass definitionsByGlossary directly to _mergeByGlossary

* Refactor how used definitions are tracked

* Change _mergeByGlossary parameters

* Simplify _mergeByGlossary

* Generate definition structure at the end of the function

* Add reusable function to clone a tag

* Restructure merged sub-definition creation

* Move setup of totalExpressionSet/totalReadingSet

* Track all definitions with matching glossary

* Move creation of definitionTags

* Update score calculation for multiple definitions

* Disambiguate function name

* Rename variables

* Add helper function _createMergedGlossaryTermDefinition

* Remove rarely used check for secondary search results

Unclear what the purpose was since it was almost never used

* Update data structure

* Rename function/variable

* Add helper functions _createMergedTermDefinition

* Assign expressionDetails to variable

* Add rawSource to grouped definitions

* Use shared function for creating unused definition entries

* Add helper function _createMapKey

* Refactor _createExpressionDetails

* Change argument order

* Add _getTagNamesWithCategory helper

* Optimize _compressDefinitionTags

* Add _createGroupedTermDefinition helper

* Add type information

* Use consistent order for term definition fields, add comments for omitted fields

* Remove unused index field

* Add helper function _createKanjiDefinition

* Update types

* Add frequencies/pitches arrays

* Update terms array setup

* Simplify for loop

* Remove redundant property assignment

* Simplify how deinflections with no definitions are ignored

* Ensure duplicates are removed

* Rename variable

* Add comments about duplicate data

* Refactor variable/parameter names

* Move functions

* Rename functions

* Rename variable

* Add dictionaryPriority field

* Simplify priority comparison

* Rename argument

* Optimize comparison function when dictionary priority is not needed

* Add length check

* Remove unused parameter

* Rename variables/parameters

* Rename dictionaries to enabledDictionaryMap

* Store compactTags/mainDictionary options to variables

* Refactor _getSearchableText
2020-10-04 11:12:15 -04:00
toasted-nutbread
ef333b6d72
Dictionary data structure improvements (#877)
* Simplify object structure of deinflections

* Modify existing array when removing duplicates

* Update _sortDefinitions to not return a value

* Use explicit argument value

* Use a Collator for string comparisons

* Simplify order comparison

* Improve tag creation and sanitization

* Switch .map to a for loop, skip null meta

* Add _createKanjiStat

* Move _sortDefinitions definition

* Fix kanji stat sorting

* Remove return value from _sortTags

* Add _sortKanji

* Add fast exits for sorting

* Add _sortDefinitionsByIndex

* Combine sort function

* Improve creation of kanji definition data

* Use "databaseDefinitions" instead of "definitions" to disambiguate

* Simplify

* Simplify further

* Simplify max length calculation

* More destructuring

* Use databaseDefinitions variable name

* Move _mergeBySequence body into _getSequencedDefinitions

* Use databaseDefinitions field name

* Move maxLength calculation

* Use shared _createTermDefinitionFromDatabaseDefinition

* Simplify map

* Move definition sorts for better consistency and less redundancy

* Move _createExpression function

* Update setup of sequenced definitions

* Simplify for loop
2020-10-02 17:59:14 -04:00
toasted-nutbread
9f8f83508e
Merge dictionary.js functions into Translator class (#726)
* Move dictFieldSplit

* Copy dictionary.js definitions into translator.js

* Convert to member functions

* Refactor names

* Remove dictionary.js

* Rename
2020-08-09 21:04:09 -04:00
toasted-nutbread
480e0e15e3
Translator refactor (#720)
* Move simple find sort into translator.js

* Remove unused

* Use direct reference to database

* Make field private

* Remove statics

* Convert functions to private

* Organize by public/private
2020-08-09 13:21:14 -04:00
toasted-nutbread
b1b33f8beb
Fix fetch requests (#708)
* Revert audio fetching functionality to use XMLHttpRequest

* Replace requestJson

* Replace requestJson

* Replace requestJson

* Replace requestJson and requestText

* Fix tests

* Include support for vulgar word searches

* Remove request.js
2020-08-02 13:30:55 -04:00
toasted-nutbread
021ccb5ac3
Move util database modification functions (#499)
* Update onProgress callback to handle multiple arguments

* Add apiImportDictionaryArchive

* Add apiDeleteDictionary

* Make onProgress the last argument for consistency

* Remove deprecated util functions

* Fix issue with missing progress args

* Remove function calls which modify the database from Translator

* Update tests

* Fix errors not being serialized correctly in _createActionListenerPort
2020-05-06 19:28:26 -04:00
toasted-nutbread
9297eb45ae
Merge pull request #450 from toasted-nutbread/frequency-readings
Add support for filtering frequency metadata based on readings
2020-04-18 14:14:30 -04:00
toasted-nutbread
649adb13d8
Merge pull request #440 from toasted-nutbread/collapse-emphatic-sequences
Add support for collapsing emphatic character sequences
2020-04-12 11:42:46 -04:00
toasted-nutbread
ceb12ac415 Add support for filtering frequency metadata based on readings 2020-04-11 23:20:36 -04:00
toasted-nutbread
70f0b8b0cd Fix 'full' mode not being a superset of 'true' mode 2020-04-11 18:58:14 -04:00
4e5029f7ec Update license author field to broader to explicitly reflect that
Yomichan is an ongoing community project. It really does take a village,
and everyone who contributes code owns a part of it :)
2020-04-10 11:10:21 -07:00
toasted-nutbread
42a2917bf7 Add support for collapsing emphatic character sequences 2020-04-10 11:56:18 -04:00
toasted-nutbread
3684a479c5
Merge pull request #424 from toasted-nutbread/dictionary-importer
Dictionary importer
2020-04-05 12:52:07 -04:00
toasted-nutbread
058f626efd
Merge pull request #423 from toasted-nutbread/text-source-map
Text source map
2020-04-05 12:51:56 -04:00
toasted-nutbread
c193a703cc Move database creation into Backend 2020-03-30 20:39:04 -04:00
toasted-nutbread
ae84d13757 Create simplified source map class 2020-03-28 18:17:00 -04:00
toasted-nutbread
047efaa3db Add support for returning pitch data from the database 2020-03-28 10:25:57 -04:00
toasted-nutbread
248a18dd72 Fix case issue 2020-03-15 22:24:30 -04:00
toasted-nutbread
0f15cca2df Convert Japanese utilities to a module-like style 2020-03-15 22:24:30 -04:00
toasted-nutbread
64fc0349a1 Update global declarations 2020-03-10 22:40:35 -04:00
toasted-nutbread
72219ba353 Replace charCodeAt and fromCharCode with codePointAt and fromCodePoint 2020-03-09 21:58:06 -04:00
siikamiika
9af8f999f4 compatibility transform non-primary definitions 2020-03-01 21:26:45 +02:00
toasted-nutbread
54bb702b84 Updates for PR feedback 2020-02-22 14:43:06 -05:00
toasted-nutbread
69b5007842 Reduce number of redundant Map.get calls in dictTermsMergeByGloss 2020-02-22 14:43:06 -05:00
toasted-nutbread
89af73b61b Use Map 2020-02-22 14:43:06 -05:00
toasted-nutbread
5e8b408a23 Change secondarySearchTitles to secondarySearchDictionaries 2020-02-22 14:43:06 -05:00
toasted-nutbread
b6718dc4a4 Use findTerms instead of findTermsInternal 2020-02-22 14:43:06 -05:00
toasted-nutbread
cc2e21cd86 Use Map for dictEnabledSet 2020-02-22 14:43:05 -05:00
toasted-nutbread
163211ade3 Use Map 2020-02-22 14:37:10 -05:00
toasted-nutbread
886278b19d Use Set 2020-02-22 14:36:07 -05:00
toasted-nutbread
0e6b75438a Use Map for Translator.tagCache 2020-02-13 20:50:30 -05:00
siikamiika
e645296b1b fix Map set in Translator.buildTermFrequencies 2020-02-14 01:12:28 +02:00
toasted-nutbread
4629fb3639 Change no-undef from off to error 2020-02-11 22:22:16 -05:00
toasted-nutbread
1d9332cb69 Simplify getSequencedDefinitions and dictTermsMergeBySequence 2020-02-01 21:00:27 -05:00
toasted-nutbread
a5fc79a754 Create buildKanjiMeta for parity with buildTermMeta 2020-01-24 22:27:25 -05:00
toasted-nutbread
8233119eb7 Update how frequency data is set up 2020-01-24 22:24:05 -05:00