Added i18n component and related scripts #1082
Conversation
…plits the text into manageable chunk
…xt handling and error reporting
…toring segment handling to use arrays for better management
…ling in Promise.all
…ity and consistency
… merging logic and remove unnecessary debug logs
…ager version in package.json
…lations inside SCHEME and SCHEMEINLINE tags
…ded try catch when parsing xml to json to report failed parsing possibly attributed to unsound xml structure.
- Created a new XML file for references (97references97.xml) containing a comprehensive list of references used in the SICP JS project. - Added a new XML file for the index preface (98indexpreface98.xml) to provide context and formatting for the index section. - Introduced a new XML file for the making section (99making99.xml) detailing the background, interactive features, and development history of the SICP JS project. - Updated subsection2.xml to close the previously open SUBSECTION tag and include a comment for clarity.
…LINE tags in XML parsing
|
breaking changes are made: xml repositories are divided into folders, currently consisting of en and cn folders to store translated content. The same applies to the json folder after running "yarn json". Frontend needs to be changed accordingly to fetch json files from the corresponding url |
i18n/controllers/translator.ts
Outdated
| baseURL: process.env.AI_BASEURL | ||
| }); | ||
|
|
||
| const MAXLEN = Number(process.env.MAX_LEN) || 3000; |
There was a problem hiding this comment.
The MAXLEN constant should be imported from the config.ts file instead of being redefined here. The config.ts already defines max_chunk_len for the same purpose with the same default value (3000).
Having configuration values defined in multiple places leads to maintenance issues and potential inconsistencies if one location is updated but not the other.
Suggestion: Replace with import { max_chunk_len } from "../config"; and use that value directly.
i18n/controllers/loggers.ts
Outdated
| console.log( | ||
| `Summary log saved to logs/translation-summary-${timestamp}.log` | ||
| ); |
There was a problem hiding this comment.
The log path is constructed differently in the code versus the message displayed to users:
// Actual path construction:
const logPath = path.join(
logDir,
`${translationSummaryPrefix}-${timestamp}.log`
);
// But message shows hardcoded format:
console.log(
`Summary log saved to logs/translation-summary-${timestamp}.log`
);
i18n/controllers/parsers.ts
Outdated
| import { Readable } from "stream"; | ||
| import { ignoredTags, max_chunk_len } from "../config"; | ||
| import fs, { PathLike } from "fs"; | ||
| import { FileLike } from "openai/uploads.mjs"; |
There was a problem hiding this comment.
The import import { FileLike } from "openai/uploads.mjs"; appears to be unused in this file. There are no references to the FileLike type throughout the code.
i18n/controllers/parsers.ts
Outdated
| import fs, { PathLike } from "fs"; | ||
| import { FileLike } from "openai/uploads.mjs"; | ||
|
|
||
| const MAXLEN = max_chunk_len; |
There was a problem hiding this comment.
The code creates an unnecessary alias for max_chunk_len. Additionally, throughout the code there are instances of Number(MAXLEN) which is redundant since max_chunk_len should already be a Number type in the config file.
i18n/initializers/initialize.ts
Outdated
| import fs from "fs"; | ||
| import OpenAI from "openai"; | ||
| import path, { dirname } from "path"; | ||
| import Stream from "stream"; |
There was a problem hiding this comment.
The Stream module is imported but never used.
translated section2 cn xmls
run yarn trans to start translation