Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Project export #3365

Merged
merged 68 commits into from
Aug 6, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
c53e062
init
ActiveChooN Jun 22, 2021
46e89b0
Fixed typos
ActiveChooN Jun 23, 2021
22fafe3
Merge branch 'develop' into dk/project-export
ActiveChooN Jun 24, 2021
7f4a349
Fixed cache time value
ActiveChooN Jun 24, 2021
4a5bc3a
Added MOTS PNG
ActiveChooN Jun 28, 2021
c57ab56
Merge branch 'develop' into dk/project-export
ActiveChooN Jul 1, 2021
04d14c5
temp
ActiveChooN Jul 2, 2021
f6b833f
Another temp
ActiveChooN Jul 7, 2021
d45bec8
WIP on UI
ActiveChooN Jul 7, 2021
b23a9ba
Added working modal
ActiveChooN Jul 8, 2021
4c22e11
Moved task export to modal
ActiveChooN Jul 9, 2021
a4d6094
Fixed vscode files
ActiveChooN Jul 9, 2021
aadc83c
Removed task actions
ActiveChooN Jul 9, 2021
bcd2034
Fixed image_maker calling
ActiveChooN Jul 12, 2021
3aa16a1
Separated DataExtractor class
ActiveChooN Jul 12, 2021
b59ab03
wip
ActiveChooN Jul 14, 2021
5d9b512
Added export with CVAT format
ActiveChooN Jul 15, 2021
a914d05
Fixed format resetting
ActiveChooN Jul 15, 2021
3b480fa
Fixed dir creating
ActiveChooN Jul 15, 2021
2647eb9
Fixed export in CVAT formats
ActiveChooN Jul 16, 2021
15d226a
Added simple server tests, fixed datumaro format export
ActiveChooN Jul 20, 2021
3da98c4
Reverted settings change
ActiveChooN Jul 20, 2021
d70ee77
Merge branch 'develop' into dk/project-export
ActiveChooN Jul 20, 2021
b8b3d9c
Fixed merge
ActiveChooN Jul 20, 2021
e1f8db6
Fixed some comments
ActiveChooN Jul 21, 2021
98c9083
Update cvat/apps/dataset_manager/bindings.py
ActiveChooN Jul 21, 2021
84f8b7a
Fixed comment
ActiveChooN Jul 21, 2021
4f8d28d
Merge branch 'dk/project-export' of https://github.com/openvinotoolki…
ActiveChooN Jul 21, 2021
29ffac1
Added restrictions for 3d tasks
ActiveChooN Jul 22, 2021
4e2aac1
Fixed validator
ActiveChooN Jul 22, 2021
5a48dfc
Merge branch 'develop' into dk/project-export
ActiveChooN Jul 23, 2021
ff19e71
Fixed linter and tests
ActiveChooN Jul 23, 2021
3e6d7df
Merge remote-tracking branch 'origin/develop' into dk/project-export
ActiveChooN Jul 26, 2021
dc45b9f
Fixed comments
ActiveChooN Jul 27, 2021
6c683ac
Added file extesion to the form
ActiveChooN Jul 27, 2021
abdb59f
Added notification
ActiveChooN Jul 27, 2021
f4c4e87
Fixed header
ActiveChooN Jul 27, 2021
76c073c
Added default format for tasks
ActiveChooN Jul 27, 2021
370f337
Fixed test
ActiveChooN Jul 28, 2021
141f6a6
Merge branch 'develop' into dk/project-export
ActiveChooN Jul 29, 2021
c076584
Fixed notification
ActiveChooN Jul 29, 2021
1f5f6f2
Added some classes
dvkruchinin Jul 29, 2021
7d1b9cf
Tests adaptation
dvkruchinin Jul 29, 2021
3be269c
Merge branch 'dk/project-export' of https://github.com/openvinotoolki…
dvkruchinin Jul 29, 2021
f129f76
Rename classes
dvkruchinin Jul 29, 2021
dc4b56a
The next tests adapdation
dvkruchinin Jul 29, 2021
cd67f5b
Additional tests adaptations.
dvkruchinin Jul 29, 2021
c955995
Some rework main.yaml
dvkruchinin Jul 29, 2021
7e9fcaa
Added debug
dvkruchinin Jul 29, 2021
4be0540
Debug removed
dvkruchinin Jul 29, 2021
415e275
Fixed late binding problem
ActiveChooN Jul 30, 2021
e6c95a2
Fixed 3d default format
ActiveChooN Jul 30, 2021
4a66693
Merge branch 'develop' into dk/project-export
ActiveChooN Jul 30, 2021
c7ed863
Merge pull request #3485 from dvkruchinin/dkru/project-export-adjast-…
ActiveChooN Jul 30, 2021
0687808
Added support for project with 3d tasks
ActiveChooN Aug 2, 2021
7e412f4
Fixed test
ActiveChooN Aug 2, 2021
4fe5598
Merge branch 'develop' into dk/project-export
ActiveChooN Aug 3, 2021
e635db3
Update cvat/apps/dataset_manager/views.py
ActiveChooN Aug 3, 2021
e9f1fc1
Fixed function name
ActiveChooN Aug 3, 2021
0a06baf
Revert "Added support for project with 3d tasks"
ActiveChooN Aug 3, 2021
1387b2f
Changed defaulted subset
ActiveChooN Aug 3, 2021
cbb7a1e
Update cvat/apps/dataset_manager/bindings.py
ActiveChooN Aug 3, 2021
566e490
Fixed project extractor
ActiveChooN Aug 3, 2021
42a974e
Merge branch 'develop' into dk/project-export
ActiveChooN Aug 4, 2021
333bc88
Fixed project export cache invalidation
ActiveChooN Aug 5, 2021
9952d51
Merge branch 'develop' into dk/project-export
ActiveChooN Aug 5, 2021
f5c89ef
Added CHANGELOG, increased versions
ActiveChooN Aug 5, 2021
a73122f
Merge branch 'develop' into dk/project-export
Aug 6, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fixed comments
  • Loading branch information
ActiveChooN committed Jul 27, 2021
commit dc45b9f73b3a25b4366017252b0814b31080e6d3
21 changes: 2 additions & 19 deletions cvat-core/src/annotations.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
const { checkObjectType } = require('./common');
const { Project } = require('./project');
const { Task, Job } = require('./session');
const { Loader, Dumper } = require('./annotation-formats');
const { Loader } = require('./annotation-formats');
const { ScriptingError, DataError, ArgumentError } = require('./exceptions');

const jobCache = new WeakMap();
Expand Down Expand Up @@ -234,22 +234,6 @@
await serverProxy.annotations.uploadAnnotations(sessionType, session.id, file, loader.name);
}

async function dumpAnnotations(session, name, dumper) {
if (!(dumper instanceof Dumper)) {
throw new ArgumentError('A dumper must be instance of Dumper class');
}

let result = null;
const sessionType = session instanceof Task ? 'task' : 'job';
if (sessionType === 'job') {
result = await serverProxy.annotations.dumpAnnotations(session.task.id, name, dumper.name);
} else {
result = await serverProxy.annotations.dumpAnnotations(session.id, name, dumper.name);
}

return result;
}

function importAnnotations(session, data) {
const sessionType = session instanceof Task ? 'task' : 'job';
const cache = getCache(sessionType);
Expand Down Expand Up @@ -282,7 +266,7 @@
throw new ArgumentError('Format must be a string');
}
if (!(instance instanceof Task || instance instanceof Project || instance instanceof Job)) {
throw new ArgumentError('A dataset can only be created from a task or project');
throw new ArgumentError('A dataset can only be created from a job, task or project');
}
if (typeof saveImages !== 'boolean') {
throw new ArgumentError('Save images parameter must be a boolean');
Expand Down Expand Up @@ -379,7 +363,6 @@
annotationsStatistics,
selectObject,
uploadAnnotations,
dumpAnnotations,
importAnnotations,
exportAnnotations,
exportDataset,
Expand Down
4 changes: 2 additions & 2 deletions cvat-core/src/api.js
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ function build() {
const Review = require('./review');
const { Job, Task } = require('./session');
const { Project } = require('./project');
require('./project-implementation');
const implementProject = require('./project-implementation');
const { Attribute, Label } = require('./labels');
const MLModel = require('./ml-model');

Expand Down Expand Up @@ -754,7 +754,7 @@ function build() {
*/
classes: {
User,
Project,
Project: implementProject(Project),
Task,
Job,
Log,
Expand Down
96 changes: 50 additions & 46 deletions cvat-core/src/project-implementation.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT

// Copyright (C) 2019-2021 Intel Corporation
ActiveChooN marked this conversation as resolved.
Show resolved Hide resolved
//
// SPDX-License-Identifier: MIT
Expand All @@ -13,58 +9,66 @@
const { Project } = require('./project');
const { exportDataset } = require('./annotations');

Project.prototype.save.implementation = async function () {
const trainingProjectCopy = this.trainingProject;
if (typeof this.id !== 'undefined') {
// project has been already created, need to update some data
const projectData = {
function implementProject(projectClass) {
projectClass.prototype.save.implementation = async function () {
const trainingProjectCopy = this.trainingProject;
if (typeof this.id !== 'undefined') {
// project has been already created, need to update some data
const projectData = {
name: this.name,
assignee_id: this.assignee ? this.assignee.id : null,
bug_tracker: this.bugTracker,
labels: [...this._internalData.labels.map((el) => el.toJSON())],
};

if (trainingProjectCopy) {
projectData.training_project = trainingProjectCopy;
}

await serverProxy.projects.save(this.id, projectData);
return this;
}

// initial creating
const projectSpec = {
name: this.name,
assignee_id: this.assignee ? this.assignee.id : null,
bug_tracker: this.bugTracker,
labels: [...this._internalData.labels.map((el) => el.toJSON())],
labels: [...this.labels.map((el) => el.toJSON())],
};

if (trainingProjectCopy) {
projectData.training_project = trainingProjectCopy;
if (this.bugTracker) {
projectSpec.bug_tracker = this.bugTracker;
}

await serverProxy.projects.save(this.id, projectData);
return this;
}
if (trainingProjectCopy) {
projectSpec.training_project = trainingProjectCopy;
}

// initial creating
const projectSpec = {
name: this.name,
labels: [...this.labels.map((el) => el.toJSON())],
const project = await serverProxy.projects.create(projectSpec);
return new Project(project);
};

if (this.bugTracker) {
projectSpec.bug_tracker = this.bugTracker;
}

if (trainingProjectCopy) {
projectSpec.training_project = trainingProjectCopy;
}
projectClass.prototype.delete.implementation = async function () {
const result = await serverProxy.projects.delete(this.id);
return result;
};

const project = await serverProxy.projects.create(projectSpec);
return new Project(project);
};
projectClass.prototype.preview.implementation = async function () {
if (!this._internalData.task_ids.length) {
return '';
}
const frameData = await getPreview(this._internalData.task_ids[0]);
return frameData;
};

Project.prototype.delete.implementation = async function () {
const result = await serverProxy.projects.delete(this.id);
return result;
};
projectClass.prototype.annotations.exportDataset.implementation = async function (
format, saveImages, customName,
) {
const result = exportDataset(this, format, customName, saveImages);
return result;
};

Project.prototype.preview.implementation = async function () {
if (!this._internalData.task_ids.length) {
return '';
}
const frameData = await getPreview(this._internalData.task_ids[0]);
return frameData;
};
return projectClass;
}

Project.prototype.annotations.exportDataset.implementation = async function (format, saveImages, customName) {
const result = exportDataset(this, format, customName, saveImages);
return result;
};
module.exports = implementProject;
})();
10 changes: 5 additions & 5 deletions cvat-core/src/project.js
Original file line number Diff line number Diff line change
Expand Up @@ -201,7 +201,7 @@
},
},
/**
* Tasks linked with the project
* Tasks related with the project
* @name tasks
* @type {module:API.cvat.classes.Task[]}
* @memberof module:API.cvat.classes.Project
Expand All @@ -212,7 +212,7 @@
get: () => [...data.tasks],
},
/**
* Subsets array for linked tasks
* Subsets array for related tasks
* @name subsets
* @type {string[]}
* @memberof module:API.cvat.classes.Project
Expand Down Expand Up @@ -253,8 +253,8 @@
}),
);

// When we call a function, for example: task.annotations.get()
// In the method get we lose the task context
// When we call a function, for example: project.annotations.get()
// In the method get we lose the project context
// So, we need return it
this.annotations = {
exportDataset: Object.getPrototypeOf(this).annotations.exportDataset.bind(this),
Expand Down Expand Up @@ -294,7 +294,7 @@
}

/**
* Method deletes a task from a server
* Method deletes a project from a server
* @method delete
* @memberof module:API.cvat.classes.Project
* @readonly
Expand Down
5 changes: 3 additions & 2 deletions cvat-core/src/server-proxy.js
Original file line number Diff line number Diff line change
Expand Up @@ -1145,8 +1145,9 @@

const closureId = Date.now();
predictAnnotations.latestRequest.id = closureId;
// eslint-disable-next-line max-len
const predicate = () => !predictAnnotations.latestRequest.fetching || predictAnnotations.latestRequest.id !== closureId;
const predicate = () => (
!predictAnnotations.latestRequest.fetching || predictAnnotations.latestRequest.id !== closureId
);
if (predictAnnotations.latestRequest.fetching) {
waitFor(5, predicate).then(() => {
if (predictAnnotations.latestRequest.id !== closureId) {
Expand Down
38 changes: 0 additions & 38 deletions cvat-core/src/session.js
Original file line number Diff line number Diff line change
Expand Up @@ -42,16 +42,6 @@
return result;
},

async dump(dumper, name = null) {
const result = await PluginRegistry.apiWrapper.call(
this,
prototype.annotations.dump,
dumper,
name,
);
return result;
},

async statistics() {
const result = await PluginRegistry.apiWrapper.call(this, prototype.annotations.statistics);
return result;
Expand Down Expand Up @@ -331,21 +321,6 @@
* @instance
* @async
*/
/**
* Dump of annotations to a file.
* Method always dumps annotations for a whole task.
* @method dump
* @memberof Session.annotations
* @param {module:API.cvat.classes.Dumper} dumper - a dumper
* @param {string} [name = null] - a name of a file with annotations
* which will be used to dump
* @returns {string} URL which can be used in order to get a dump file
* @throws {module:API.cvat.exceptions.PluginError}
* @throws {module:API.cvat.exceptions.ServerError}
* @throws {module:API.cvat.exceptions.ArgumentError}
* @instance
* @async
*/
/**
* Collect short statistics about a task or a job.
* @method statistics
Expand Down Expand Up @@ -879,7 +854,6 @@
get: Object.getPrototypeOf(this).annotations.get.bind(this),
put: Object.getPrototypeOf(this).annotations.put.bind(this),
save: Object.getPrototypeOf(this).annotations.save.bind(this),
dump: Object.getPrototypeOf(this).annotations.dump.bind(this),
merge: Object.getPrototypeOf(this).annotations.merge.bind(this),
split: Object.getPrototypeOf(this).annotations.split.bind(this),
group: Object.getPrototypeOf(this).annotations.group.bind(this),
Expand Down Expand Up @@ -1577,7 +1551,6 @@
get: Object.getPrototypeOf(this).annotations.get.bind(this),
put: Object.getPrototypeOf(this).annotations.put.bind(this),
save: Object.getPrototypeOf(this).annotations.save.bind(this),
dump: Object.getPrototypeOf(this).annotations.dump.bind(this),
merge: Object.getPrototypeOf(this).annotations.merge.bind(this),
split: Object.getPrototypeOf(this).annotations.split.bind(this),
group: Object.getPrototypeOf(this).annotations.group.bind(this),
Expand Down Expand Up @@ -1717,7 +1690,6 @@
selectObject,
annotationsStatistics,
uploadAnnotations,
dumpAnnotations,
importAnnotations,
exportAnnotations,
exportDataset,
Expand Down Expand Up @@ -1950,11 +1922,6 @@
return result;
};

Job.prototype.annotations.dump.implementation = async function (dumper, name) {
const result = await dumpAnnotations(this, name, dumper);
return result;
};

Job.prototype.annotations.exportDataset.implementation = async function (format, saveImages, customName) {
const result = await exportDataset(this.task, format, customName, saveImages);
return result;
Expand Down Expand Up @@ -2254,11 +2221,6 @@
return result;
};

Task.prototype.annotations.dump.implementation = async function (dumper, name) {
const result = await dumpAnnotations(this, name, dumper);
return result;
};

Task.prototype.annotations.import.implementation = function (data) {
const result = importAnnotations(this, data);
return result;
Expand Down
17 changes: 8 additions & 9 deletions cvat-ui/src/actions/export-actions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,14 @@ export enum ExportActionTypes {
export const exportActions = {
openExportModal: (instance: any) => createAction(ExportActionTypes.OPEN_EXPORT_MODAL, { instance }),
closeExportModal: () => createAction(ExportActionTypes.CLOSE_EXPORT_MODAL),
exportDataset: (instance: any, format: any, saveImages: boolean) =>
createAction(ExportActionTypes.EXPORT_DATASET, { instance, format, saveImages }),
exportDatasetSuccess: (instance: any, format: any, saveImages: boolean) =>
createAction(ExportActionTypes.EXPORT_DATASET_SUCCESS, { instance, format, saveImages }),
exportDatasetFailed: (instance: any, format: any, saveImages: boolean, error: any) =>
exportDataset: (instance: any, format: string) =>
createAction(ExportActionTypes.EXPORT_DATASET, { instance, format }),
exportDatasetSuccess: (instance: any, format: string) =>
createAction(ExportActionTypes.EXPORT_DATASET_SUCCESS, { instance, format }),
exportDatasetFailed: (instance: any, format: string, error: any) =>
createAction(ExportActionTypes.EXPORT_DATASET_FAILED, {
instance,
format,
saveImages,
error,
}),
};
Expand All @@ -34,16 +33,16 @@ export const exportDatasetAsync = (
name: string,
saveImages: boolean,
): ThunkAction => async (dispatch) => {
dispatch(exportActions.exportDataset(instance, format, saveImages));
dispatch(exportActions.exportDataset(instance, format));

try {
const url = await instance.annotations.exportDataset(format, saveImages, name);
const downloadAnchor = window.document.getElementById('downloadAnchor') as HTMLAnchorElement;
downloadAnchor.href = url;
downloadAnchor.click();
dispatch(exportActions.exportDatasetSuccess(instance, format, saveImages));
dispatch(exportActions.exportDatasetSuccess(instance, format));
} catch (error) {
dispatch(exportActions.exportDatasetFailed(instance, format, saveImages, error));
dispatch(exportActions.exportDatasetFailed(instance, format, error));
}
};

Expand Down
6 changes: 2 additions & 4 deletions cvat-ui/src/components/actions-menu/actions-menu.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,6 @@ interface Props {
loaders: any[];
dumpers: any[];
loadActivity: string | null;
dumpActivities: string[] | null;
exportActivities: string[] | null;
inferenceIsActive: boolean;
taskDimension: DimensionType;
onClickMenu: (params: MenuInfo, file?: File) => void;
Expand Down Expand Up @@ -108,14 +106,14 @@ export default function ActionsMenuComponent(props: Props): JSX.Element {
menuKey: Actions.LOAD_TASK_ANNO,
taskDimension,
})}
<Menu.Item key={Actions.EXPORT_TASK_DATASET}>Export Task dataset</Menu.Item>
<Menu.Item key={Actions.EXPORT_TASK_DATASET}>Export task dataset</Menu.Item>
{!!bugTracker && <Menu.Item key={Actions.OPEN_BUG_TRACKER}>Open bug tracker</Menu.Item>}
<Menu.Item disabled={inferenceIsActive} key={Actions.RUN_AUTO_ANNOTATION}>
Automatic annotation
</Menu.Item>
<Menu.Item key={Actions.EXPORT_TASK} disabled={exportIsActive}>
{exportIsActive && <LoadingOutlined id='cvat-export-task-loading' />}
Export Task
Export task
</Menu.Item>
<hr />
<Menu.Item key={Actions.MOVE_TASK_TO_PROJECT}>Move to project</Menu.Item>
Expand Down
Loading