Skip to content

Commit

Permalink
Remove dataset parameter for orca-mini-3b (openvinotoolkit#760)
Browse files Browse the repository at this point in the history 
Since AWQ is disabled and ratio is 1.0, there is no need for statistics
collection.

Related to huggingface/optimum-intel#862
  • Loading branch information
@nikita-savelyevv
nikita-savelyevv authored Aug 12, 2024
1 parent 5f78871 commit 5ec8926
Showing 1 changed file with 1 addition and 2 deletions.
3 changes: 1 addition & 2 deletions llm_bench/python/utils/nncf_utils.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -60,8 +60,7 @@ def get_compressed_path(output_dir: str, base_precision, option: str):
"open-llama-3b": {"mode": nncf.CompressWeightsMode.INT4_ASYM, "group_size": 64, "ratio": 1.0, "all_layers": True},
"open-llama-3b-v2": {"mode": nncf.CompressWeightsMode.INT4_ASYM, "group_size": 64, "ratio": 1.0},
"falcon-7b-instruct": {"mode": nncf.CompressWeightsMode.INT4_SYM, "group_size": 64, "all_layers": True},
"orca-mini-3b": {"mode": nncf.CompressWeightsMode.INT4_SYM, "group_size": 64, "all_layers": True,
"dataset": {"name": "wikitext,wikitext-2-v1,train[:1000],text", "awq": False}},
"orca-mini-3b": {"mode": nncf.CompressWeightsMode.INT4_SYM, "group_size": 64, "all_layers": True},
"bloomz-560m": {"mode": nncf.CompressWeightsMode.INT4_SYM, "group_size": 64, "ratio": 0.8,
"dataset": {"name": "wikitext,wikitext-2-v1,train[:1000],text", "awq": True}},
"mixtral-8x7b-v0.1": {"mode": nncf.CompressWeightsMode.INT4_SYM, "group_size": 128, "ratio": 0.8},
Expand Down

0 comments on commit 5ec8926

Please sign in to comment.