Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: fix #1807 #1837

Merged
merged 2 commits into from
Oct 6, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion integration-tests/snapshots/AdaDoom3/AdaDoom3/report.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Repository: AdaDoom3/AdaDoom3
Url: https://github.com/AdaDoom3/AdaDoom3.git
Args: ["**/*.*"]
Summary:
files: 103
files: 105
filesWithIssues: 98
issues: 3265
errors: 0
Expand Down
2 changes: 1 addition & 1 deletion integration-tests/snapshots/AdaDoom3/AdaDoom3/snapshot.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Repository: AdaDoom3/AdaDoom3
Url: "https://github.com/AdaDoom3/AdaDoom3.git"
Args: ["**/*.*"]
Lines:
CSpell: Files checked: 103, Issues found: 3265 in 98 files
CSpell: Files checked: 105, Issues found: 3265 in 98 files
exit code: 1
./Engine/Assembly/PPC/neo-engine-cpu.adb:5:78 - Unknown word (Squirek) -- Copyright (C) 2016 Justin Squirek
./Engine/Assembly/x86-64/neo-engine-cpu.adb:101:111 - Unknown word (OSXSAVE) -- ECX_Register, 27); -- OSXSAVE
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Repository: Azure/azure-rest-api-specs
Url: "https://github.com/Azure/azure-rest-api-specs.git"
Args: ["--config=cSpell.json","**/*.{md,ts,js}"]
Lines:
CSpell: Files checked: 1678, Issues found: 1721 in 584 files
CSpell: Files checked: 1679, Issues found: 1721 in 584 files
exit code: 1
./README.md:20:179 - Unknown word (Dataplane) -- RPs' Resource types or Dataplane API versions that represent
./README.md:30:134 - Unknown word (dataplane) -- per resource type or dataplane service version. This
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Repository: SoftwareBrothers/admin-bro
Url: "https://github.com/SoftwareBrothers/admin-bro.git"
Args: ["src/**/*.{ts,js,tsx,jsx}","**/*.md"]
Lines:
CSpell: Files checked: 360, Issues found: 6 in 5 files
CSpell: Files checked: 361, Issues found: 6 in 5 files
exit code: 1
./cy/readme.md:433:45 - Unknown word (Favourite) -- Sequelize Resources” → Favourite Places link inside the
./src/backend/utils/build-feature/build-feature.spec.ts:86:14 - Unknown word (falsey) -- it('merges falsey options', function
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Repository: TheAlgorithms/Python
Url: https://github.com/TheAlgorithms/Python.git
Args: ["**/*.{md,py}"]
Summary:
files: 783
files: 939
filesWithIssues: 395
issues: 2537
errors: 0
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Repository: TheAlgorithms/Python
Url: "https://github.com/TheAlgorithms/Python.git"
Args: ["**/*.{md,py}"]
Lines:
CSpell: Files checked: 783, Issues found: 2537 in 395 files
CSpell: Files checked: 939, Issues found: 2537 in 395 files
exit code: 1
./CONTRIBUTING.md:131:33 - Unknown word (pytest) -- doctests will be run by pytest as part of our automated
./CONTRIBUTING.md:134:14 - Unknown word (doctest) -- python3 -m doctest -v my_submission.py
Expand Down
2 changes: 1 addition & 1 deletion integration-tests/snapshots/django/django/report.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Repository: django/django
Url: https://github.com/django/django.git
Args: ["**/*.{md,py}"]
Summary:
files: 2132
files: 2701
filesWithIssues: 878
issues: 7789
errors: 0
Expand Down
2 changes: 1 addition & 1 deletion integration-tests/snapshots/django/django/snapshot.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Repository: django/django
Url: "https://github.com/django/django.git"
Args: ["**/*.{md,py}"]
Lines:
CSpell: Files checked: 2132, Issues found: 7789 in 878 files
CSpell: Files checked: 2701, Issues found: 7789 in 878 files
exit code: 1
./django/apps/config.py:125:76 - Unknown word (isclass) -- getmembers(mod, inspect.isclass)
./django/apps/registry.py:132:40 - Unknown word (unconfigured) -- not ready" is due to unconfigured settings, accessing
Expand Down
2 changes: 1 addition & 1 deletion integration-tests/snapshots/liriliri/licia/snapshot.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Repository: liriliri/licia
Url: "https://github.com/liriliri/licia.git"
Args: ["**/*.*"]
Lines:
CSpell: Files checked: 1362, Issues found: 353 in 86 files
CSpell: Files checked: 1365, Issues found: 353 in 86 files
exit code: 1
./CHANGELOG.md:227:26 - Unknown word (unenumerable) -- feat(allKeys): support unenumerable and symbol
./CHANGELOG.md:85:22 - Unknown word (eval) -- fix(Class): unsafe-eval CSP violation
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Repository: microsoft/TypeScript-Website
Url: https://github.com/microsoft/TypeScript-Website.git
Args: ["--config=${repoConfig}/cspell.json","**/*.*"]
Summary:
files: 705
files: 706
filesWithIssues: 149
issues: 571
errors: 0
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Repository: microsoft/TypeScript-Website
Url: "https://github.com/microsoft/TypeScript-Website.git"
Args: ["--config=../../../../config/repositories/microsoft/TypeScript-Website/cspell.json","**/*.*"]
Lines:
CSpell: Files checked: 705, Issues found: 571 in 149 files
CSpell: Files checked: 706, Issues found: 571 in 149 files
exit code: 1
./docs/Deprecating a page.md:3:109 - Unknown word (Dont) -- deleting because [Cool URIs Dont Change](https://www
./docs/New TypeScript Version.md:115:67 - Unknown word (schemastore) -- schemastore.git /tmp/schemastore
Expand Down
2 changes: 1 addition & 1 deletion integration-tests/snapshots/php/php-src/report.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Repository: php/php-src
Url: https://github.com/php/php-src.git
Args: ["--config=${repoConfig}/cspell.json","**/*.{md,c,h,php}"]
Summary:
files: 1826
files: 1827
filesWithIssues: 1106
issues: 19948
errors: 0
Expand Down
2 changes: 1 addition & 1 deletion integration-tests/snapshots/php/php-src/snapshot.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Repository: php/php-src
Url: "https://github.com/php/php-src.git"
Args: ["--config=../../../../config/repositories/php/php-src/cspell.json","**/*.{md,c,h,php}"]
Lines:
CSpell: Files checked: 1826, Issues found: 19948 in 1106 files
CSpell: Files checked: 1827, Issues found: 19948 in 1106 files
exit code: 1
./CODING_STANDARDS.md:105:8 - Unknown word (setclientencoding) -- pg_setclientencoding
./CODING_STANDARDS.md:126:5 - Unknown word (fooselect) -- fooselect_bar
Expand Down
2 changes: 1 addition & 1 deletion integration-tests/snapshots/prettier/prettier/snapshot.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,5 @@ Repository: prettier/prettier
Url: "https://github.com/prettier/prettier.git"
Args: []
Lines:
CSpell: Files checked: 1733, Issues found: 0 in 0 files
CSpell: Files checked: 1740, Issues found: 0 in 0 files
exit code: 0
2 changes: 1 addition & 1 deletion integration-tests/snapshots/pycontribs/jira/report.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Repository: pycontribs/jira
Url: https://github.com/pycontribs/jira.git
Args: ["**/*.*"]
Summary:
files: 69
files: 76
filesWithIssues: 36
issues: 459
errors: 0
Expand Down
2 changes: 1 addition & 1 deletion integration-tests/snapshots/pycontribs/jira/snapshot.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Repository: pycontribs/jira
Url: "https://github.com/pycontribs/jira.git"
Args: ["**/*.*"]
Lines:
CSpell: Files checked: 69, Issues found: 459 in 36 files
CSpell: Files checked: 76, Issues found: 459 in 36 files
exit code: 1
./AUTHORS.rst:3:21 - Unknown word (Contribs) -- Development Team (PyContribs)
./AUTHORS.rst:5:7 - Unknown word (Speakmon) -- - Ben Speakmon <ben.speakmon@gmail
Expand Down
1 change: 1 addition & 0 deletions packages/cspell-types/src/CSpellReporter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ export interface ProgressFileComplete extends ProgressBase {
elapsedTimeMs: number | undefined;
processed: boolean | undefined;
numErrors: number | undefined;
cached?: boolean;
}

export type ProgressEmitter = (p: ProgressItem | ProgressFileComplete) => void;
Expand Down
5 changes: 3 additions & 2 deletions packages/cspell/src/cli-reporter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -66,13 +66,14 @@ function reportProgress(p: ProgressItem) {
const fn = (' '.repeat(fc.length) + p.fileNum).slice(-fc.length);
const idx = fn + '/' + fc;
const filename = chalk.gray(relativeFilename(p.filename));
const time = reportTime(p.elapsedTimeMs);
const time = reportTime(p.elapsedTimeMs, !!p.cached);
const skipped = p.processed === false ? ' skipped' : '';
const hasErrors = p.numErrors ? chalk.red` X` : '';
console.error(`${idx} ${filename} ${time}${skipped}${hasErrors}`);
}

function reportTime(elapsedTimeMs: number | undefined): string {
function reportTime(elapsedTimeMs: number | undefined, cached: boolean): string {
if (cached) return chalk.green('cached');
if (elapsedTimeMs === undefined) return '-';
const color = elapsedTimeMs < 1000 ? chalk.white : elapsedTimeMs < 2000 ? chalk.yellow : chalk.redBright;
return color(elapsedTimeMs.toFixed(2) + 'ms');
Expand Down
3 changes: 2 additions & 1 deletion packages/cspell/src/fileHelper.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,8 @@ export interface FileResult {
issues: Issue[];
errors: number;
configErrors: number;
elapsedTimeMs: number;
elapsedTimeMs: number | undefined;
cached?: boolean;
}

export function fileInfoToDocument(
Expand Down
10 changes: 6 additions & 4 deletions packages/cspell/src/lint.ts
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ export async function runLint(cfg: CSpellApplicationConfiguration): Promise<RunR
const status: RunResult = runResult();
const cache = createCache(cfg);

const emitProgress = (filename: string, fileNum: number, result?: FileResult) =>
const emitProgress = (filename: string, fileNum: number, result: FileResult) =>
reporter.progress({
type: 'ProgressFileComplete',
fileNum,
Expand All @@ -108,6 +108,7 @@ export async function runLint(cfg: CSpellApplicationConfiguration): Promise<RunR
elapsedTimeMs: result?.elapsedTimeMs,
processed: result?.processed,
numErrors: result?.issues.length,
cached: result?.cached,
});

async function* loadAndProcessFiles() {
Expand All @@ -120,15 +121,16 @@ export async function runLint(cfg: CSpellApplicationConfiguration): Promise<RunR

for await (const fileP of loadAndProcessFiles()) {
const { filename, fileNum, result } = await fileP;
if (!result.fileInfo.text) {
emitProgress(filename, fileNum);
if (!result.fileInfo.text === undefined) {
status.files += result.cached ? 1 : 0;
emitProgress(filename, fileNum, result);
continue;
}

status.files += 1;
emitProgress(filename, fileNum, result);
// Show the spelling errors after emitting the progress.
result.issues.filter(cfg.uniqueFilter).forEach((issue) => reporter.issue(issue));
status.files += 1;
if (result.issues.length || result.errors) {
status.filesWithIssues.add(filename);
status.issues += result.issues.length;
Expand Down
9 changes: 6 additions & 3 deletions packages/cspell/src/util/cache/DiskCache.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,8 @@ describe('DiskCache', () => {
const cachedResult = await diskCache.getCachedLintResults('file', TEST_CONFIG__INFO);

expect(cachedResult).toMatchObject(RESULT_NO_ISSUES);
expect(cachedResult?.elapsedTimeMs).toEqual(0);
expect(cachedResult?.elapsedTimeMs).toBeUndefined();
expect(cachedResult?.cached).toBe(true);

expect(mockReadFileInfo).toHaveBeenCalledTimes(0);
expect(cachedResult?.fileInfo.filename).toEqual('file');
Expand All @@ -94,7 +95,8 @@ describe('DiskCache', () => {
const cachedResult = await diskCache.getCachedLintResults('file', TEST_CONFIG__INFO);

expect(cachedResult).toMatchObject(RESULT_NO_ISSUES);
expect(cachedResult?.elapsedTimeMs).toEqual(0);
expect(cachedResult?.elapsedTimeMs).toBeUndefined();
expect(cachedResult?.cached).toBe(false);

expect(mockReadFileInfo).toHaveBeenCalledTimes(0);
expect(cachedResult?.fileInfo.filename).toEqual('file');
Expand All @@ -116,7 +118,8 @@ describe('DiskCache', () => {
const cachedResult = await diskCache.getCachedLintResults('file', TEST_CONFIG__INFO);

expect(cachedResult).toMatchObject(result);
expect(cachedResult?.elapsedTimeMs).toEqual(0);
expect(cachedResult?.elapsedTimeMs).toBeUndefined();
expect(cachedResult?.cached).toBe(true);

expect(mockReadFileInfo).toHaveBeenCalledTimes(1);
expect(cachedResult?.fileInfo).toEqual(fileInfo);
Expand Down
6 changes: 4 additions & 2 deletions packages/cspell/src/util/cache/DiskCache.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,12 +44,14 @@ export class DiskCache implements CSpellLintResultCache {

// Skip reading empty files and files without lint error
const hasErrors = meta.result.errors > 0 || meta.result.configErrors > 0 || meta.result.issues.length > 0;
const shouldReadFile = meta.size !== 0 && hasErrors;
const cached = !!meta.size;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be called nonEmpty for readbility?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is communicating to the cache consumer that the entry was cached. How that calculation is done is up to the cache itself. It is possible that cached = true might be the correct thing and restore the original shouldReadFile calculation. It is still not clear to me under which situations meta.size will be 0.

const shouldReadFile = cached && hasErrors;

return {
...meta.result,
elapsedTimeMs: 0,
elapsedTimeMs: undefined,
fileInfo: shouldReadFile ? await readFileInfo(filename) : { filename },
cached,
};
}

Expand Down