Hugging Face's logo

Abhisek987
/
phi35-vision-pdf-markdown

Text Generation
PEFT
Safetensors
Transformers
phi3_v
lora
custom_code
Model card Files
xet
 Community
Abhisek987 commited on
Commit
497edcb
·
verified ·
1 Parent(s): 9a75b62

Upload folder using huggingface_hub

Browse files
Files changed (46) hide show
  1. README.md +207 -0
  2. adapter_config.json +43 -0
  3. adapter_model.safetensors +3 -0
  4. checkpoint-500/README.md +207 -0
  5. checkpoint-500/adapter_config.json +43 -0
  6. checkpoint-500/adapter_model.safetensors +3 -0
  7. checkpoint-500/chat_template.json +3 -0
  8. checkpoint-500/non_lora_state_dict.bin +3 -0
  9. checkpoint-500/preprocessor_config.json +21 -0
  10. checkpoint-500/special_tokens_map.json +30 -0
  11. checkpoint-500/tokenizer.json +0 -0
  12. checkpoint-500/tokenizer_config.json +414 -0
  13. checkpoint-500/training_args.bin +3 -0
  14. checkpoint-5000/README.md +207 -0
  15. checkpoint-5000/adapter_config.json +43 -0
  16. checkpoint-5000/adapter_model.safetensors +3 -0
  17. checkpoint-5000/chat_template.json +3 -0
  18. checkpoint-5000/non_lora_state_dict.bin +3 -0
  19. checkpoint-5000/optimizer.pt +3 -0
  20. checkpoint-5000/preprocessor_config.json +21 -0
  21. checkpoint-5000/rng_state.pth +3 -0
  22. checkpoint-5000/scheduler.pt +3 -0
  23. checkpoint-5000/special_tokens_map.json +30 -0
  24. checkpoint-5000/tokenizer.json +0 -0
  25. checkpoint-5000/tokenizer_config.json +414 -0
  26. checkpoint-5000/trainer_state.json +3533 -0
  27. checkpoint-5000/training_args.bin +3 -0
  28. checkpoint-5914/README.md +207 -0
  29. checkpoint-5914/adapter_config.json +43 -0
  30. checkpoint-5914/adapter_model.safetensors +3 -0
  31. checkpoint-5914/chat_template.json +3 -0
  32. checkpoint-5914/non_lora_state_dict.bin +3 -0
  33. checkpoint-5914/optimizer.pt +3 -0
  34. checkpoint-5914/preprocessor_config.json +21 -0
  35. checkpoint-5914/rng_state.pth +3 -0
  36. checkpoint-5914/scheduler.pt +3 -0
  37. checkpoint-5914/special_tokens_map.json +30 -0
  38. checkpoint-5914/tokenizer.json +0 -0
  39. checkpoint-5914/tokenizer_config.json +414 -0
  40. checkpoint-5914/trainer_state.json +0 -0
  41. checkpoint-5914/training_args.bin +3 -0
  42. config.json +155 -0
  43. non_lora_state_dict.bin +3 -0
  44. trainer_state.json +0 -0
  45. training_log_stage7.txt +0 -0
  46. training_log_stage7_v2.txt +0 -0
README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: microsoft/Phi-3.5-vision-instruct
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:microsoft/Phi-3.5-vision-instruct
7
+ - lora
8
+ - transformers
9
+ ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+
16
+
17
+ ## Model Details
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+
24
+
25
+ - **Developed by:** [More Information Needed]
26
+ - **Funded by [optional]:** [More Information Needed]
27
+ - **Shared by [optional]:** [More Information Needed]
28
+ - **Model type:** [More Information Needed]
29
+ - **Language(s) (NLP):** [More Information Needed]
30
+ - **License:** [More Information Needed]
31
+ - **Finetuned from model [optional]:** [More Information Needed]
32
+
33
+ ### Model Sources [optional]
34
+
35
+ <!-- Provide the basic links for the model. -->
36
+
37
+ - **Repository:** [More Information Needed]
38
+ - **Paper [optional]:** [More Information Needed]
39
+ - **Demo [optional]:** [More Information Needed]
40
+
41
+ ## Uses
42
+
43
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
44
+
45
+ ### Direct Use
46
+
47
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
48
+
49
+ [More Information Needed]
50
+
51
+ ### Downstream Use [optional]
52
+
53
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
54
+
55
+ [More Information Needed]
56
+
57
+ ### Out-of-Scope Use
58
+
59
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
60
+
61
+ [More Information Needed]
62
+
63
+ ## Bias, Risks, and Limitations
64
+
65
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
66
+
67
+ [More Information Needed]
68
+
69
+ ### Recommendations
70
+
71
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
72
+
73
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
74
+
75
+ ## How to Get Started with the Model
76
+
77
+ Use the code below to get started with the model.
78
+
79
+ [More Information Needed]
80
+
81
+ ## Training Details
82
+
83
+ ### Training Data
84
+
85
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
86
+
87
+ [More Information Needed]
88
+
89
+ ### Training Procedure
90
+
91
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
92
+
93
+ #### Preprocessing [optional]
94
+
95
+ [More Information Needed]
96
+
97
+
98
+ #### Training Hyperparameters
99
+
100
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
101
+
102
+ #### Speeds, Sizes, Times [optional]
103
+
104
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
105
+
106
+ [More Information Needed]
107
+
108
+ ## Evaluation
109
+
110
+ <!-- This section describes the evaluation protocols and provides the results. -->
111
+
112
+ ### Testing Data, Factors & Metrics
113
+
114
+ #### Testing Data
115
+
116
+ <!-- This should link to a Dataset Card if possible. -->
117
+
118
+ [More Information Needed]
119
+
120
+ #### Factors
121
+
122
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
123
+
124
+ [More Information Needed]
125
+
126
+ #### Metrics
127
+
128
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
129
+
130
+ [More Information Needed]
131
+
132
+ ### Results
133
+
134
+ [More Information Needed]
135
+
136
+ #### Summary
137
+
138
+
139
+
140
+ ## Model Examination [optional]
141
+
142
+ <!-- Relevant interpretability work for the model goes here -->
143
+
144
+ [More Information Needed]
145
+
146
+ ## Environmental Impact
147
+
148
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
149
+
150
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
151
+
152
+ - **Hardware Type:** [More Information Needed]
153
+ - **Hours used:** [More Information Needed]
154
+ - **Cloud Provider:** [More Information Needed]
155
+ - **Compute Region:** [More Information Needed]
156
+ - **Carbon Emitted:** [More Information Needed]
157
+
158
+ ## Technical Specifications [optional]
159
+
160
+ ### Model Architecture and Objective
161
+
162
+ [More Information Needed]
163
+
164
+ ### Compute Infrastructure
165
+
166
+ [More Information Needed]
167
+
168
+ #### Hardware
169
+
170
+ [More Information Needed]
171
+
172
+ #### Software
173
+
174
+ [More Information Needed]
175
+
176
+ ## Citation [optional]
177
+
178
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
179
+
180
+ **BibTeX:**
181
+
182
+ [More Information Needed]
183
+
184
+ **APA:**
185
+
186
+ [More Information Needed]
187
+
188
+ ## Glossary [optional]
189
+
190
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
191
+
192
+ [More Information Needed]
193
+
194
+ ## More Information [optional]
195
+
196
+ [More Information Needed]
197
+
198
+ ## Model Card Authors [optional]
199
+
200
+ [More Information Needed]
201
+
202
+ ## Model Card Contact
203
+
204
+ [More Information Needed]
205
+ ### Framework versions
206
+
207
+ - PEFT 0.18.1
adapter_config.json ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "microsoft/Phi-3.5-vision-instruct",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": false,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "layer_replication": null,
16
+ "layers_pattern": null,
17
+ "layers_to_transform": null,
18
+ "loftq_config": {},
19
+ "lora_alpha": 128,
20
+ "lora_bias": false,
21
+ "lora_dropout": 0.05,
22
+ "megatron_config": null,
23
+ "megatron_core": "megatron.core",
24
+ "modules_to_save": null,
25
+ "peft_type": "LORA",
26
+ "peft_version": "0.18.1",
27
+ "qalora_group_size": 16,
28
+ "r": 64,
29
+ "rank_pattern": {},
30
+ "revision": null,
31
+ "target_modules": [
32
+ "down_proj",
33
+ "qkv_proj",
34
+ "gate_up_proj",
35
+ "o_proj"
36
+ ],
37
+ "target_parameters": null,
38
+ "task_type": "CAUSAL_LM",
39
+ "trainable_token_indices": null,
40
+ "use_dora": false,
41
+ "use_qalora": false,
42
+ "use_rslora": false
43
+ }
adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d67c56376e70283e205d2850989edf7759aa866cebc1c9d59eba61dcc9830d97
3
+ size 402688040
checkpoint-500/README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: microsoft/Phi-3.5-vision-instruct
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:microsoft/Phi-3.5-vision-instruct
7
+ - lora
8
+ - transformers
9
+ ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+
16
+
17
+ ## Model Details
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+
24
+
25
+ - **Developed by:** [More Information Needed]
26
+ - **Funded by [optional]:** [More Information Needed]
27
+ - **Shared by [optional]:** [More Information Needed]
28
+ - **Model type:** [More Information Needed]
29
+ - **Language(s) (NLP):** [More Information Needed]
30
+ - **License:** [More Information Needed]
31
+ - **Finetuned from model [optional]:** [More Information Needed]
32
+
33
+ ### Model Sources [optional]
34
+
35
+ <!-- Provide the basic links for the model. -->
36
+
37
+ - **Repository:** [More Information Needed]
38
+ - **Paper [optional]:** [More Information Needed]
39
+ - **Demo [optional]:** [More Information Needed]
40
+
41
+ ## Uses
42
+
43
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
44
+
45
+ ### Direct Use
46
+
47
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
48
+
49
+ [More Information Needed]
50
+
51
+ ### Downstream Use [optional]
52
+
53
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
54
+
55
+ [More Information Needed]
56
+
57
+ ### Out-of-Scope Use
58
+
59
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
60
+
61
+ [More Information Needed]
62
+
63
+ ## Bias, Risks, and Limitations
64
+
65
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
66
+
67
+ [More Information Needed]
68
+
69
+ ### Recommendations
70
+
71
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
72
+
73
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
74
+
75
+ ## How to Get Started with the Model
76
+
77
+ Use the code below to get started with the model.
78
+
79
+ [More Information Needed]
80
+
81
+ ## Training Details
82
+
83
+ ### Training Data
84
+
85
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
86
+
87
+ [More Information Needed]
88
+
89
+ ### Training Procedure
90
+
91
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
92
+
93
+ #### Preprocessing [optional]
94
+
95
+ [More Information Needed]
96
+
97
+
98
+ #### Training Hyperparameters
99
+
100
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
101
+
102
+ #### Speeds, Sizes, Times [optional]
103
+
104
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
105
+
106
+ [More Information Needed]
107
+
108
+ ## Evaluation
109
+
110
+ <!-- This section describes the evaluation protocols and provides the results. -->
111
+
112
+ ### Testing Data, Factors & Metrics
113
+
114
+ #### Testing Data
115
+
116
+ <!-- This should link to a Dataset Card if possible. -->
117
+
118
+ [More Information Needed]
119
+
120
+ #### Factors
121
+
122
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
123
+
124
+ [More Information Needed]
125
+
126
+ #### Metrics
127
+
128
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
129
+
130
+ [More Information Needed]
131
+
132
+ ### Results
133
+
134
+ [More Information Needed]
135
+
136
+ #### Summary
137
+
138
+
139
+
140
+ ## Model Examination [optional]
141
+
142
+ <!-- Relevant interpretability work for the model goes here -->
143
+
144
+ [More Information Needed]
145
+
146
+ ## Environmental Impact
147
+
148
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
149
+
150
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
151
+
152
+ - **Hardware Type:** [More Information Needed]
153
+ - **Hours used:** [More Information Needed]
154
+ - **Cloud Provider:** [More Information Needed]
155
+ - **Compute Region:** [More Information Needed]
156
+ - **Carbon Emitted:** [More Information Needed]
157
+
158
+ ## Technical Specifications [optional]
159
+
160
+ ### Model Architecture and Objective
161
+
162
+ [More Information Needed]
163
+
164
+ ### Compute Infrastructure
165
+
166
+ [More Information Needed]
167
+
168
+ #### Hardware
169
+
170
+ [More Information Needed]
171
+
172
+ #### Software
173
+
174
+ [More Information Needed]
175
+
176
+ ## Citation [optional]
177
+
178
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
179
+
180
+ **BibTeX:**
181
+
182
+ [More Information Needed]
183
+
184
+ **APA:**
185
+
186
+ [More Information Needed]
187
+
188
+ ## Glossary [optional]
189
+
190
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
191
+
192
+ [More Information Needed]
193
+
194
+ ## More Information [optional]
195
+
196
+ [More Information Needed]
197
+
198
+ ## Model Card Authors [optional]
199
+
200
+ [More Information Needed]
201
+
202
+ ## Model Card Contact
203
+
204
+ [More Information Needed]
205
+ ### Framework versions
206
+
207
+ - PEFT 0.18.1
checkpoint-500/adapter_config.json ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "microsoft/Phi-3.5-vision-instruct",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": false,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "layer_replication": null,
16
+ "layers_pattern": null,
17
+ "layers_to_transform": null,
18
+ "loftq_config": {},
19
+ "lora_alpha": 128,
20
+ "lora_bias": false,
21
+ "lora_dropout": 0.05,
22
+ "megatron_config": null,
23
+ "megatron_core": "megatron.core",
24
+ "modules_to_save": null,
25
+ "peft_type": "LORA",
26
+ "peft_version": "0.18.1",
27
+ "qalora_group_size": 16,
28
+ "r": 64,
29
+ "rank_pattern": {},
30
+ "revision": null,
31
+ "target_modules": [
32
+ "gate_up_proj",
33
+ "down_proj",
34
+ "o_proj",
35
+ "qkv_proj"
36
+ ],
37
+ "target_parameters": null,
38
+ "task_type": "CAUSAL_LM",
39
+ "trainable_token_indices": null,
40
+ "use_dora": false,
41
+ "use_qalora": false,
42
+ "use_rslora": false
43
+ }
checkpoint-500/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6b28c7d065c2c9a5a6021f123456c529889f76f654d921831dcc7d4979acdf28
3
+ size 402688040
checkpoint-500/chat_template.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "chat_template": "{% for message in messages %}{{'<|' + message['role'] + '|>' + '\n' + message['content'] + '<|end|>\n' }}{% endfor %}{% if add_generation_prompt and messages[-1]['role'] != 'assistant' %}{{- '<|assistant|>\n' -}}{% endif %}"
3
+ }
checkpoint-500/non_lora_state_dict.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:72b88f54449e1c9615bd0114d417394af1282fa1329c6cc4aa509d7648d72f5c
3
+ size 1925120128
checkpoint-500/preprocessor_config.json ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "auto_map": {
3
+ "AutoImageProcessor": "microsoft/Phi-3.5-vision-instruct--processing_phi3_v.Phi3VImageProcessor",
4
+ "AutoProcessor": "microsoft/Phi-3.5-vision-instruct--processing_phi3_v.Phi3VProcessor"
5
+ },
6
+ "do_convert_rgb": true,
7
+ "image_mean": [
8
+ 0.48145466,
9
+ 0.4578275,
10
+ 0.40821073
11
+ ],
12
+ "image_processor_type": "Phi3VImageProcessor",
13
+ "image_std": [
14
+ 0.26862954,
15
+ 0.26130258,
16
+ 0.27577711
17
+ ],
18
+ "num_crops": 16,
19
+ "num_img_tokens": 144,
20
+ "processor_class": "Phi3VProcessor"
21
+ }
checkpoint-500/special_tokens_map.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "additional_special_tokens": [
3
+ "<|system|>",
4
+ "<|end|>",
5
+ "<|user|>",
6
+ "<|end|>"
7
+ ],
8
+ "bos_token": {
9
+ "content": "<s>",
10
+ "lstrip": false,
11
+ "normalized": false,
12
+ "rstrip": false,
13
+ "single_word": false
14
+ },
15
+ "eos_token": {
16
+ "content": "<|endoftext|>",
17
+ "lstrip": false,
18
+ "normalized": false,
19
+ "rstrip": false,
20
+ "single_word": false
21
+ },
22
+ "pad_token": "<unk>",
23
+ "unk_token": {
24
+ "content": "<unk>",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ }
30
+ }
checkpoint-500/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-500/tokenizer_config.json ADDED
@@ -0,0 +1,414 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_bos_token": true,
3
+ "add_eos_token": false,
4
+ "add_prefix_space": null,
5
+ "added_tokens_decoder": {
6
+ "0": {
7
+ "content": "<unk>",
8
+ "lstrip": false,
9
+ "normalized": false,
10
+ "rstrip": false,
11
+ "single_word": false,
12
+ "special": true
13
+ },
14
+ "1": {
15
+ "content": "<s>",
16
+ "lstrip": false,
17
+ "normalized": false,
18
+ "rstrip": false,
19
+ "single_word": false,
20
+ "special": true
21
+ },
22
+ "2": {
23
+ "content": "</s>",
24
+ "lstrip": false,
25
+ "normalized": false,
26
+ "rstrip": true,
27
+ "single_word": false,
28
+ "special": false
29
+ },
30
+ "32000": {
31
+ "content": "<|endoftext|>",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false,
36
+ "special": true
37
+ },
38
+ "32001": {
39
+ "content": "<|assistant|>",
40
+ "lstrip": false,
41
+ "normalized": false,
42
+ "rstrip": true,
43
+ "single_word": false,
44
+ "special": true
45
+ },
46
+ "32002": {
47
+ "content": "<|placeholder1|>",
48
+ "lstrip": false,
49
+ "normalized": false,
50
+ "rstrip": true,
51
+ "single_word": false,
52
+ "special": true
53
+ },
54
+ "32003": {
55
+ "content": "<|placeholder2|>",
56
+ "lstrip": false,
57
+ "normalized": false,
58
+ "rstrip": true,
59
+ "single_word": false,
60
+ "special": true
61
+ },
62
+ "32004": {
63
+ "content": "<|placeholder3|>",
64
+ "lstrip": false,
65
+ "normalized": false,
66
+ "rstrip": true,
67
+ "single_word": false,
68
+ "special": true
69
+ },
70
+ "32005": {
71
+ "content": "<|placeholder4|>",
72
+ "lstrip": false,
73
+ "normalized": false,
74
+ "rstrip": true,
75
+ "single_word": false,
76
+ "special": true
77
+ },
78
+ "32006": {
79
+ "content": "<|system|>",
80
+ "lstrip": false,
81
+ "normalized": false,
82
+ "rstrip": false,
83
+ "single_word": false,
84
+ "special": true
85
+ },
86
+ "32007": {
87
+ "content": "<|end|>",
88
+ "lstrip": false,
89
+ "normalized": false,
90
+ "rstrip": false,
91
+ "single_word": false,
92
+ "special": true
93
+ },
94
+ "32008": {
95
+ "content": "<|placeholder5|>",
96
+ "lstrip": false,
97
+ "normalized": false,
98
+ "rstrip": true,
99
+ "single_word": false,
100
+ "special": true
101
+ },
102
+ "32009": {
103
+ "content": "<|placeholder6|>",
104
+ "lstrip": false,
105
+ "normalized": false,
106
+ "rstrip": true,
107
+ "single_word": false,
108
+ "special": true
109
+ },
110
+ "32010": {
111
+ "content": "<|user|>",
112
+ "lstrip": false,
113
+ "normalized": false,
114
+ "rstrip": false,
115
+ "single_word": false,
116
+ "special": true
117
+ },
118
+ "32011": {
119
+ "content": "<|placeholder7|>",
120
+ "lstrip": false,
121
+ "normalized": false,
122
+ "rstrip": true,
123
+ "single_word": false,
124
+ "special": true
125
+ },
126
+ "32012": {
127
+ "content": "<|placeholder8|>",
128
+ "lstrip": false,
129
+ "normalized": false,
130
+ "rstrip": true,
131
+ "single_word": false,
132
+ "special": true
133
+ },
134
+ "32013": {
135
+ "content": "<|placeholder9|>",
136
+ "lstrip": false,
137
+ "normalized": false,
138
+ "rstrip": true,
139
+ "single_word": false,
140
+ "special": true
141
+ },
142
+ "32014": {
143
+ "content": "<|placeholder10|>",
144
+ "lstrip": false,
145
+ "normalized": false,
146
+ "rstrip": true,
147
+ "single_word": false,
148
+ "special": true
149
+ },
150
+ "32015": {
151
+ "content": "<|placeholder11|>",
152
+ "lstrip": false,
153
+ "normalized": false,
154
+ "rstrip": true,
155
+ "single_word": false,
156
+ "special": true
157
+ },
158
+ "32016": {
159
+ "content": "<|placeholder12|>",
160
+ "lstrip": false,
161
+ "normalized": false,
162
+ "rstrip": true,
163
+ "single_word": false,
164
+ "special": true
165
+ },
166
+ "32017": {
167
+ "content": "<|placeholder13|>",
168
+ "lstrip": false,
169
+ "normalized": false,
170
+ "rstrip": true,
171
+ "single_word": false,
172
+ "special": true
173
+ },
174
+ "32018": {
175
+ "content": "<|placeholder14|>",
176
+ "lstrip": false,
177
+ "normalized": false,
178
+ "rstrip": true,
179
+ "single_word": false,
180
+ "special": true
181
+ },
182
+ "32019": {
183
+ "content": "<|placeholder15|>",
184
+ "lstrip": false,
185
+ "normalized": false,
186
+ "rstrip": true,
187
+ "single_word": false,
188
+ "special": true
189
+ },
190
+ "32020": {
191
+ "content": "<|placeholder16|>",
192
+ "lstrip": false,
193
+ "normalized": false,
194
+ "rstrip": true,
195
+ "single_word": false,
196
+ "special": true
197
+ },
198
+ "32021": {
199
+ "content": "<|placeholder17|>",
200
+ "lstrip": false,
201
+ "normalized": false,
202
+ "rstrip": true,
203
+ "single_word": false,
204
+ "special": true
205
+ },
206
+ "32022": {
207
+ "content": "<|placeholder18|>",
208
+ "lstrip": false,
209
+ "normalized": false,
210
+ "rstrip": true,
211
+ "single_word": false,
212
+ "special": true
213
+ },
214
+ "32023": {
215
+ "content": "<|placeholder19|>",
216
+ "lstrip": false,
217
+ "normalized": false,
218
+ "rstrip": true,
219
+ "single_word": false,
220
+ "special": true
221
+ },
222
+ "32024": {
223
+ "content": "<|placeholder20|>",
224
+ "lstrip": false,
225
+ "normalized": false,
226
+ "rstrip": true,
227
+ "single_word": false,
228
+ "special": true
229
+ },
230
+ "32025": {
231
+ "content": "<|placeholder21|>",
232
+ "lstrip": false,
233
+ "normalized": false,
234
+ "rstrip": true,
235
+ "single_word": false,
236
+ "special": true
237
+ },
238
+ "32026": {
239
+ "content": "<|placeholder22|>",
240
+ "lstrip": false,
241
+ "normalized": false,
242
+ "rstrip": true,
243
+ "single_word": false,
244
+ "special": true
245
+ },
246
+ "32027": {
247
+ "content": "<|placeholder23|>",
248
+ "lstrip": false,
249
+ "normalized": false,
250
+ "rstrip": true,
251
+ "single_word": false,
252
+ "special": true
253
+ },
254
+ "32028": {
255
+ "content": "<|placeholder24|>",
256
+ "lstrip": false,
257
+ "normalized": false,
258
+ "rstrip": true,
259
+ "single_word": false,
260
+ "special": true
261
+ },
262
+ "32029": {
263
+ "content": "<|placeholder25|>",
264
+ "lstrip": false,
265
+ "normalized": false,
266
+ "rstrip": true,
267
+ "single_word": false,
268
+ "special": true
269
+ },
270
+ "32030": {
271
+ "content": "<|placeholder26|>",
272
+ "lstrip": false,
273
+ "normalized": false,
274
+ "rstrip": true,
275
+ "single_word": false,
276
+ "special": true
277
+ },
278
+ "32031": {
279
+ "content": "<|placeholder27|>",
280
+ "lstrip": false,
281
+ "normalized": false,
282
+ "rstrip": true,
283
+ "single_word": false,
284
+ "special": true
285
+ },
286
+ "32032": {
287
+ "content": "<|placeholder28|>",
288
+ "lstrip": false,
289
+ "normalized": false,
290
+ "rstrip": true,
291
+ "single_word": false,
292
+ "special": true
293
+ },
294
+ "32033": {
295
+ "content": "<|placeholder29|>",
296
+ "lstrip": false,
297
+ "normalized": false,
298
+ "rstrip": true,
299
+ "single_word": false,
300
+ "special": true
301
+ },
302
+ "32034": {
303
+ "content": "<|placeholder30|>",
304
+ "lstrip": false,
305
+ "normalized": false,
306
+ "rstrip": true,
307
+ "single_word": false,
308
+ "special": true
309
+ },
310
+ "32035": {
311
+ "content": "<|placeholder31|>",
312
+ "lstrip": false,
313
+ "normalized": false,
314
+ "rstrip": true,
315
+ "single_word": false,
316
+ "special": true
317
+ },
318
+ "32036": {
319
+ "content": "<|placeholder32|>",
320
+ "lstrip": false,
321
+ "normalized": false,
322
+ "rstrip": true,
323
+ "single_word": false,
324
+ "special": true
325
+ },
326
+ "32037": {
327
+ "content": "<|placeholder33|>",
328
+ "lstrip": false,
329
+ "normalized": false,
330
+ "rstrip": true,
331
+ "single_word": false,
332
+ "special": true
333
+ },
334
+ "32038": {
335
+ "content": "<|placeholder34|>",
336
+ "lstrip": false,
337
+ "normalized": false,
338
+ "rstrip": true,
339
+ "single_word": false,
340
+ "special": true
341
+ },
342
+ "32039": {
343
+ "content": "<|placeholder35|>",
344
+ "lstrip": false,
345
+ "normalized": false,
346
+ "rstrip": true,
347
+ "single_word": false,
348
+ "special": true
349
+ },
350
+ "32040": {
351
+ "content": "<|placeholder36|>",
352
+ "lstrip": false,
353
+ "normalized": false,
354
+ "rstrip": true,
355
+ "single_word": false,
356
+ "special": true
357
+ },
358
+ "32041": {
359
+ "content": "<|placeholder37|>",
360
+ "lstrip": false,
361
+ "normalized": false,
362
+ "rstrip": true,
363
+ "single_word": false,
364
+ "special": true
365
+ },
366
+ "32042": {
367
+ "content": "<|placeholder38|>",
368
+ "lstrip": false,
369
+ "normalized": false,
370
+ "rstrip": true,
371
+ "single_word": false,
372
+ "special": true
373
+ },
374
+ "32043": {
375
+ "content": "<|placeholder39|>",
376
+ "lstrip": false,
377
+ "normalized": false,
378
+ "rstrip": true,
379
+ "single_word": false,
380
+ "special": true
381
+ },
382
+ "32044": {
383
+ "content": "<|image|>",
384
+ "lstrip": false,
385
+ "normalized": false,
386
+ "rstrip": true,
387
+ "single_word": false,
388
+ "special": true
389
+ }
390
+ },
391
+ "additional_special_tokens": [
392
+ "<|system|>",
393
+ "<|end|>",
394
+ "<|user|>",
395
+ "<|end|>"
396
+ ],
397
+ "auto_map": {
398
+ "AutoProcessor": "microsoft/Phi-3.5-vision-instruct--processing_phi3_v.Phi3VProcessor"
399
+ },
400
+ "bos_token": "<s>",
401
+ "chat_template": "{% for message in messages %}{{'<|' + message['role'] + '|>' + '\n' + message['content'] + '<|end|>\n' }}{% endfor %}{% if add_generation_prompt and messages[-1]['role'] != 'assistant' %}{{- '<|assistant|>\n' -}}{% endif %}",
402
+ "clean_up_tokenization_spaces": false,
403
+ "eos_token": "<|endoftext|>",
404
+ "legacy": false,
405
+ "model_max_length": 2048,
406
+ "num_crops": 16,
407
+ "pad_token": "<unk>",
408
+ "padding_side": "right",
409
+ "processor_class": "Phi3VProcessor",
410
+ "sp_model_kwargs": {},
411
+ "tokenizer_class": "LlamaTokenizer",
412
+ "unk_token": "<unk>",
413
+ "use_default_system_prompt": false
414
+ }
checkpoint-500/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1989d97e020c2d179384d8387e24b4d26fe5288ae8c3d293f4ee4f4d8371e73b
3
+ size 6289
checkpoint-5000/README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: microsoft/Phi-3.5-vision-instruct
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:microsoft/Phi-3.5-vision-instruct
7
+ - lora
8
+ - transformers
9
+ ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+
16
+
17
+ ## Model Details
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+
24
+
25
+ - **Developed by:** [More Information Needed]
26
+ - **Funded by [optional]:** [More Information Needed]
27
+ - **Shared by [optional]:** [More Information Needed]
28
+ - **Model type:** [More Information Needed]
29
+ - **Language(s) (NLP):** [More Information Needed]
30
+ - **License:** [More Information Needed]
31
+ - **Finetuned from model [optional]:** [More Information Needed]
32
+
33
+ ### Model Sources [optional]
34
+
35
+ <!-- Provide the basic links for the model. -->
36
+
37
+ - **Repository:** [More Information Needed]
38
+ - **Paper [optional]:** [More Information Needed]
39
+ - **Demo [optional]:** [More Information Needed]
40
+
41
+ ## Uses
42
+
43
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
44
+
45
+ ### Direct Use
46
+
47
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
48
+
49
+ [More Information Needed]
50
+
51
+ ### Downstream Use [optional]
52
+
53
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
54
+
55
+ [More Information Needed]
56
+
57
+ ### Out-of-Scope Use
58
+
59
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
60
+
61
+ [More Information Needed]
62
+
63
+ ## Bias, Risks, and Limitations
64
+
65
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
66
+
67
+ [More Information Needed]
68
+
69
+ ### Recommendations
70
+
71
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
72
+
73
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
74
+
75
+ ## How to Get Started with the Model
76
+
77
+ Use the code below to get started with the model.
78
+
79
+ [More Information Needed]
80
+
81
+ ## Training Details
82
+
83
+ ### Training Data
84
+
85
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
86
+
87
+ [More Information Needed]
88
+
89
+ ### Training Procedure
90
+
91
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
92
+
93
+ #### Preprocessing [optional]
94
+
95
+ [More Information Needed]
96
+
97
+
98
+ #### Training Hyperparameters
99
+
100
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
101
+
102
+ #### Speeds, Sizes, Times [optional]
103
+
104
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
105
+
106
+ [More Information Needed]
107
+
108
+ ## Evaluation
109
+
110
+ <!-- This section describes the evaluation protocols and provides the results. -->
111
+
112
+ ### Testing Data, Factors & Metrics
113
+
114
+ #### Testing Data
115
+
116
+ <!-- This should link to a Dataset Card if possible. -->
117
+
118
+ [More Information Needed]
119
+
120
+ #### Factors
121
+
122
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
123
+
124
+ [More Information Needed]
125
+
126
+ #### Metrics
127
+
128
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
129
+
130
+ [More Information Needed]
131
+
132
+ ### Results
133
+
134
+ [More Information Needed]
135
+
136
+ #### Summary
137
+
138
+
139
+
140
+ ## Model Examination [optional]
141
+
142
+ <!-- Relevant interpretability work for the model goes here -->
143
+
144
+ [More Information Needed]
145
+
146
+ ## Environmental Impact
147
+
148
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
149
+
150
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
151
+
152
+ - **Hardware Type:** [More Information Needed]
153
+ - **Hours used:** [More Information Needed]
154
+ - **Cloud Provider:** [More Information Needed]
155
+ - **Compute Region:** [More Information Needed]
156
+ - **Carbon Emitted:** [More Information Needed]
157
+
158
+ ## Technical Specifications [optional]
159
+
160
+ ### Model Architecture and Objective
161
+
162
+ [More Information Needed]
163
+
164
+ ### Compute Infrastructure
165
+
166
+ [More Information Needed]
167
+
168
+ #### Hardware
169
+
170
+ [More Information Needed]
171
+
172
+ #### Software
173
+
174
+ [More Information Needed]
175
+
176
+ ## Citation [optional]
177
+
178
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
179
+
180
+ **BibTeX:**
181
+
182
+ [More Information Needed]
183
+
184
+ **APA:**
185
+
186
+ [More Information Needed]
187
+
188
+ ## Glossary [optional]
189
+
190
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
191
+
192
+ [More Information Needed]
193
+
194
+ ## More Information [optional]
195
+
196
+ [More Information Needed]
197
+
198
+ ## Model Card Authors [optional]
199
+
200
+ [More Information Needed]
201
+
202
+ ## Model Card Contact
203
+
204
+ [More Information Needed]
205
+ ### Framework versions
206
+
207
+ - PEFT 0.18.1
checkpoint-5000/adapter_config.json ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "microsoft/Phi-3.5-vision-instruct",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": false,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "layer_replication": null,
16
+ "layers_pattern": null,
17
+ "layers_to_transform": null,
18
+ "loftq_config": {},
19
+ "lora_alpha": 128,
20
+ "lora_bias": false,
21
+ "lora_dropout": 0.05,
22
+ "megatron_config": null,
23
+ "megatron_core": "megatron.core",
24
+ "modules_to_save": null,
25
+ "peft_type": "LORA",
26
+ "peft_version": "0.18.1",
27
+ "qalora_group_size": 16,
28
+ "r": 64,
29
+ "rank_pattern": {},
30
+ "revision": null,
31
+ "target_modules": [
32
+ "down_proj",
33
+ "qkv_proj",
34
+ "gate_up_proj",
35
+ "o_proj"
36
+ ],
37
+ "target_parameters": null,
38
+ "task_type": "CAUSAL_LM",
39
+ "trainable_token_indices": null,
40
+ "use_dora": false,
41
+ "use_qalora": false,
42
+ "use_rslora": false
43
+ }
checkpoint-5000/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ccbc6b73d5b1eeb56a846a7e1ba083d07d62a5bd2ebf3290f632cb0de4e45348
3
+ size 402688040
checkpoint-5000/chat_template.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "chat_template": "{% for message in messages %}{{'<|' + message['role'] + '|>' + '\n' + message['content'] + '<|end|>\n' }}{% endfor %}{% if add_generation_prompt and messages[-1]['role'] != 'assistant' %}{{- '<|assistant|>\n' -}}{% endif %}"
3
+ }
checkpoint-5000/non_lora_state_dict.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2043c04dab255ee33d8177cc5145d0395d45cb9af4bc0bf51ea58b5c7b901448
3
+ size 8293470415
checkpoint-5000/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4d98c7d70862ce4015eae957f203782ba486b73d7575db06073ecf60bd4e0f98
3
+ size 805522635
checkpoint-5000/preprocessor_config.json ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "auto_map": {
3
+ "AutoImageProcessor": "microsoft/Phi-3.5-vision-instruct--processing_phi3_v.Phi3VImageProcessor",
4
+ "AutoProcessor": "microsoft/Phi-3.5-vision-instruct--processing_phi3_v.Phi3VProcessor"
5
+ },
6
+ "do_convert_rgb": true,
7
+ "image_mean": [
8
+ 0.48145466,
9
+ 0.4578275,
10
+ 0.40821073
11
+ ],
12
+ "image_processor_type": "Phi3VImageProcessor",
13
+ "image_std": [
14
+ 0.26862954,
15
+ 0.26130258,
16
+ 0.27577711
17
+ ],
18
+ "num_crops": 16,
19
+ "num_img_tokens": 144,
20
+ "processor_class": "Phi3VProcessor"
21
+ }
checkpoint-5000/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d7dbfba8589a01cd25123fbec09a7e47349b2fbf2f7d5280b3277929477b7c04
3
+ size 14709
checkpoint-5000/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:48ebc4dd20d19be85fcf4f032b26ce20dd5806659a46c0805f7779a4da194401
3
+ size 1465
checkpoint-5000/special_tokens_map.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "additional_special_tokens": [
3
+ "<|system|>",
4
+ "<|end|>",
5
+ "<|user|>",
6
+ "<|end|>"
7
+ ],
8
+ "bos_token": {
9
+ "content": "<s>",
10
+ "lstrip": false,
11
+ "normalized": false,
12
+ "rstrip": false,
13
+ "single_word": false
14
+ },
15
+ "eos_token": {
16
+ "content": "<|endoftext|>",
17
+ "lstrip": false,
18
+ "normalized": false,
19
+ "rstrip": false,
20
+ "single_word": false
21
+ },
22
+ "pad_token": "<unk>",
23
+ "unk_token": {
24
+ "content": "<unk>",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ }
30
+ }
checkpoint-5000/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-5000/tokenizer_config.json ADDED
@@ -0,0 +1,414 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_bos_token": true,
3
+ "add_eos_token": false,
4
+ "add_prefix_space": null,
5
+ "added_tokens_decoder": {
6
+ "0": {
7
+ "content": "<unk>",
8
+ "lstrip": false,
9
+ "normalized": false,
10
+ "rstrip": false,
11
+ "single_word": false,
12
+ "special": true
13
+ },
14
+ "1": {
15
+ "content": "<s>",
16
+ "lstrip": false,
17
+ "normalized": false,
18
+ "rstrip": false,
19
+ "single_word": false,
20
+ "special": true
21
+ },
22
+ "2": {
23
+ "content": "</s>",
24
+ "lstrip": false,
25
+ "normalized": false,
26
+ "rstrip": true,
27
+ "single_word": false,
28
+ "special": false
29
+ },
30
+ "32000": {
31
+ "content": "<|endoftext|>",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false,
36
+ "special": true
37
+ },
38
+ "32001": {
39
+ "content": "<|assistant|>",
40
+ "lstrip": false,
41
+ "normalized": false,
42
+ "rstrip": true,
43
+ "single_word": false,
44
+ "special": true
45
+ },
46
+ "32002": {
47
+ "content": "<|placeholder1|>",
48
+ "lstrip": false,
49
+ "normalized": false,
50
+ "rstrip": true,
51
+ "single_word": false,
52
+ "special": true
53
+ },
54
+ "32003": {
55
+ "content": "<|placeholder2|>",
56
+ "lstrip": false,
57
+ "normalized": false,
58
+ "rstrip": true,
59
+ "single_word": false,
60
+ "special": true
61
+ },
62
+ "32004": {
63
+ "content": "<|placeholder3|>",
64
+ "lstrip": false,
65
+ "normalized": false,
66
+ "rstrip": true,
67
+ "single_word": false,
68
+ "special": true
69
+ },
70
+ "32005": {
71
+ "content": "<|placeholder4|>",
72
+ "lstrip": false,
73
+ "normalized": false,
74
+ "rstrip": true,
75
+ "single_word": false,
76
+ "special": true
77
+ },
78
+ "32006": {
79
+ "content": "<|system|>",
80
+ "lstrip": false,
81
+ "normalized": false,
82
+ "rstrip": false,
83
+ "single_word": false,
84
+ "special": true
85
+ },
86
+ "32007": {
87
+ "content": "<|end|>",
88
+ "lstrip": false,
89
+ "normalized": false,
90
+ "rstrip": false,
91
+ "single_word": false,
92
+ "special": true
93
+ },
94
+ "32008": {
95
+ "content": "<|placeholder5|>",
96
+ "lstrip": false,
97
+ "normalized": false,
98
+ "rstrip": true,
99
+ "single_word": false,
100
+ "special": true
101
+ },
102
+ "32009": {
103
+ "content": "<|placeholder6|>",
104
+ "lstrip": false,
105
+ "normalized": false,
106
+ "rstrip": true,
107
+ "single_word": false,
108
+ "special": true
109
+ },
110
+ "32010": {
111
+ "content": "<|user|>",
112
+ "lstrip": false,
113
+ "normalized": false,
114
+ "rstrip": false,
115
+ "single_word": false,
116
+ "special": true
117
+ },
118
+ "32011": {
119
+ "content": "<|placeholder7|>",
120
+ "lstrip": false,
121
+ "normalized": false,
122
+ "rstrip": true,
123
+ "single_word": false,
124
+ "special": true
125
+ },
126
+ "32012": {
127
+ "content": "<|placeholder8|>",
128
+ "lstrip": false,
129
+ "normalized": false,
130
+ "rstrip": true,
131
+ "single_word": false,
132
+ "special": true
133
+ },
134
+ "32013": {
135
+ "content": "<|placeholder9|>",
136
+ "lstrip": false,
137
+ "normalized": false,
138
+ "rstrip": true,
139
+ "single_word": false,
140
+ "special": true
141
+ },
142
+ "32014": {
143
+ "content": "<|placeholder10|>",
144
+ "lstrip": false,
145
+ "normalized": false,
146
+ "rstrip": true,
147
+ "single_word": false,
148
+ "special": true
149
+ },
150
+ "32015": {
151
+ "content": "<|placeholder11|>",
152
+ "lstrip": false,
153
+ "normalized": false,
154
+ "rstrip": true,
155
+ "single_word": false,
156
+ "special": true
157
+ },
158
+ "32016": {
159
+ "content": "<|placeholder12|>",
160
+ "lstrip": false,
161
+ "normalized": false,
162
+ "rstrip": true,
163
+ "single_word": false,
164
+ "special": true
165
+ },
166
+ "32017": {
167
+ "content": "<|placeholder13|>",
168
+ "lstrip": false,
169
+ "normalized": false,
170
+ "rstrip": true,
171
+ "single_word": false,
172
+ "special": true
173
+ },
174
+ "32018": {
175
+ "content": "<|placeholder14|>",
176
+ "lstrip": false,
177
+ "normalized": false,
178
+ "rstrip": true,
179
+ "single_word": false,
180
+ "special": true
181
+ },
182
+ "32019": {
183
+ "content": "<|placeholder15|>",
184
+ "lstrip": false,
185
+ "normalized": false,
186
+ "rstrip": true,
187
+ "single_word": false,
188
+ "special": true
189
+ },
190
+ "32020": {
191
+ "content": "<|placeholder16|>",
192
+ "lstrip": false,
193
+ "normalized": false,
194
+ "rstrip": true,
195
+ "single_word": false,
196
+ "special": true
197
+ },
198
+ "32021": {
199
+ "content": "<|placeholder17|>",
200
+ "lstrip": false,
201
+ "normalized": false,
202
+ "rstrip": true,
203
+ "single_word": false,
204
+ "special": true
205
+ },
206
+ "32022": {
207
+ "content": "<|placeholder18|>",
208
+ "lstrip": false,
209
+ "normalized": false,
210
+ "rstrip": true,
211
+ "single_word": false,
212
+ "special": true
213
+ },
214
+ "32023": {
215
+ "content": "<|placeholder19|>",
216
+ "lstrip": false,
217
+ "normalized": false,
218
+ "rstrip": true,
219
+ "single_word": false,
220
+ "special": true
221
+ },
222
+ "32024": {
223
+ "content": "<|placeholder20|>",
224
+ "lstrip": false,
225
+ "normalized": false,
226
+ "rstrip": true,
227
+ "single_word": false,
228
+ "special": true
229
+ },
230
+ "32025": {
231
+ "content": "<|placeholder21|>",
232
+ "lstrip": false,
233
+ "normalized": false,
234
+ "rstrip": true,
235
+ "single_word": false,
236
+ "special": true
237
+ },
238
+ "32026": {
239
+ "content": "<|placeholder22|>",
240
+ "lstrip": false,
241
+ "normalized": false,
242
+ "rstrip": true,
243
+ "single_word": false,
244
+ "special": true
245
+ },
246
+ "32027": {
247
+ "content": "<|placeholder23|>",
248
+ "lstrip": false,
249
+ "normalized": false,
250
+ "rstrip": true,
251
+ "single_word": false,
252
+ "special": true
253
+ },
254
+ "32028": {
255
+ "content": "<|placeholder24|>",
256
+ "lstrip": false,
257
+ "normalized": false,
258
+ "rstrip": true,
259
+ "single_word": false,
260
+ "special": true
261
+ },
262
+ "32029": {
263
+ "content": "<|placeholder25|>",
264
+ "lstrip": false,
265
+ "normalized": false,
266
+ "rstrip": true,
267
+ "single_word": false,
268
+ "special": true
269
+ },
270
+ "32030": {
271
+ "content": "<|placeholder26|>",
272
+ "lstrip": false,
273
+ "normalized": false,
274
+ "rstrip": true,
275
+ "single_word": false,
276
+ "special": true
277
+ },
278
+ "32031": {
279
+ "content": "<|placeholder27|>",
280
+ "lstrip": false,
281
+ "normalized": false,
282
+ "rstrip": true,
283
+ "single_word": false,
284
+ "special": true
285
+ },
286
+ "32032": {
287
+ "content": "<|placeholder28|>",
288
+ "lstrip": false,
289
+ "normalized": false,
290
+ "rstrip": true,
291
+ "single_word": false,
292
+ "special": true
293
+ },
294
+ "32033": {
295
+ "content": "<|placeholder29|>",
296
+ "lstrip": false,
297
+ "normalized": false,
298
+ "rstrip": true,
299
+ "single_word": false,
300
+ "special": true
301
+ },
302
+ "32034": {
303
+ "content": "<|placeholder30|>",
304
+ "lstrip": false,
305
+ "normalized": false,
306
+ "rstrip": true,
307
+ "single_word": false,
308
+ "special": true
309
+ },
310
+ "32035": {
311
+ "content": "<|placeholder31|>",
312
+ "lstrip": false,
313
+ "normalized": false,
314
+ "rstrip": true,
315
+ "single_word": false,
316
+ "special": true
317
+ },
318
+ "32036": {
319
+ "content": "<|placeholder32|>",
320
+ "lstrip": false,
321
+ "normalized": false,
322
+ "rstrip": true,
323
+ "single_word": false,
324
+ "special": true
325
+ },
326
+ "32037": {
327
+ "content": "<|placeholder33|>",
328
+ "lstrip": false,
329
+ "normalized": false,
330
+ "rstrip": true,
331
+ "single_word": false,
332
+ "special": true
333
+ },
334
+ "32038": {
335
+ "content": "<|placeholder34|>",
336
+ "lstrip": false,
337
+ "normalized": false,
338
+ "rstrip": true,
339
+ "single_word": false,
340
+ "special": true
341
+ },
342
+ "32039": {
343
+ "content": "<|placeholder35|>",
344
+ "lstrip": false,
345
+ "normalized": false,
346
+ "rstrip": true,
347
+ "single_word": false,
348
+ "special": true
349
+ },
350
+ "32040": {
351
+ "content": "<|placeholder36|>",
352
+ "lstrip": false,
353
+ "normalized": false,
354
+ "rstrip": true,
355
+ "single_word": false,
356
+ "special": true
357
+ },
358
+ "32041": {
359
+ "content": "<|placeholder37|>",
360
+ "lstrip": false,
361
+ "normalized": false,
362
+ "rstrip": true,
363
+ "single_word": false,
364
+ "special": true
365
+ },
366
+ "32042": {
367
+ "content": "<|placeholder38|>",
368
+ "lstrip": false,
369
+ "normalized": false,
370
+ "rstrip": true,
371
+ "single_word": false,
372
+ "special": true
373
+ },
374
+ "32043": {
375
+ "content": "<|placeholder39|>",
376
+ "lstrip": false,
377
+ "normalized": false,
378
+ "rstrip": true,
379
+ "single_word": false,
380
+ "special": true
381
+ },
382
+ "32044": {
383
+ "content": "<|image|>",
384
+ "lstrip": false,
385
+ "normalized": false,
386
+ "rstrip": true,
387
+ "single_word": false,
388
+ "special": true
389
+ }
390
+ },
391
+ "additional_special_tokens": [
392
+ "<|system|>",
393
+ "<|end|>",
394
+ "<|user|>",
395
+ "<|end|>"
396
+ ],
397
+ "auto_map": {
398
+ "AutoProcessor": "microsoft/Phi-3.5-vision-instruct--processing_phi3_v.Phi3VProcessor"
399
+ },
400
+ "bos_token": "<s>",
401
+ "chat_template": "{% for message in messages %}{{'<|' + message['role'] + '|>' + '\n' + message['content'] + '<|end|>\n' }}{% endfor %}{% if add_generation_prompt and messages[-1]['role'] != 'assistant' %}{{- '<|assistant|>\n' -}}{% endif %}",
402
+ "clean_up_tokenization_spaces": false,
403
+ "eos_token": "<|endoftext|>",
404
+ "legacy": false,
405
+ "model_max_length": 2048,
406
+ "num_crops": 16,
407
+ "pad_token": "<unk>",
408
+ "padding_side": "right",
409
+ "processor_class": "Phi3VProcessor",
410
+ "sp_model_kwargs": {},
411
+ "tokenizer_class": "LlamaTokenizer",
412
+ "unk_token": "<unk>",
413
+ "use_default_system_prompt": false
414
+ }
checkpoint-5000/trainer_state.json ADDED
@@ -0,0 +1,3533 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": null,
3
+ "best_model_checkpoint": null,
4
+ "epoch": 1.690474178006931,
5
+ "eval_steps": 500,
6
+ "global_step": 5000,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.003380948356013862,
13
+ "grad_norm": 0.8766934275627136,
14
+ "learning_rate": 5.617977528089888e-07,
15
+ "loss": 0.9149,
16
+ "step": 10
17
+ },
18
+ {
19
+ "epoch": 0.006761896712027724,
20
+ "grad_norm": 0.8079432249069214,
21
+ "learning_rate": 1.1235955056179777e-06,
22
+ "loss": 0.9489,
23
+ "step": 20
24
+ },
25
+ {
26
+ "epoch": 0.010142845068041586,
27
+ "grad_norm": 0.7274302244186401,
28
+ "learning_rate": 1.6853932584269663e-06,
29
+ "loss": 0.9124,
30
+ "step": 30
31
+ },
32
+ {
33
+ "epoch": 0.013523793424055448,
34
+ "grad_norm": 0.7323054671287537,
35
+ "learning_rate": 2.2471910112359554e-06,
36
+ "loss": 0.8207,
37
+ "step": 40
38
+ },
39
+ {
40
+ "epoch": 0.01690474178006931,
41
+ "grad_norm": 0.5992118716239929,
42
+ "learning_rate": 2.8089887640449444e-06,
43
+ "loss": 0.9128,
44
+ "step": 50
45
+ },
46
+ {
47
+ "epoch": 0.02028569013608317,
48
+ "grad_norm": 0.5319453477859497,
49
+ "learning_rate": 3.3707865168539327e-06,
50
+ "loss": 0.7734,
51
+ "step": 60
52
+ },
53
+ {
54
+ "epoch": 0.023666638492097034,
55
+ "grad_norm": 0.5841428637504578,
56
+ "learning_rate": 3.932584269662922e-06,
57
+ "loss": 0.8069,
58
+ "step": 70
59
+ },
60
+ {
61
+ "epoch": 0.027047586848110896,
62
+ "grad_norm": 0.6458873748779297,
63
+ "learning_rate": 4.494382022471911e-06,
64
+ "loss": 0.6812,
65
+ "step": 80
66
+ },
67
+ {
68
+ "epoch": 0.03042853520412476,
69
+ "grad_norm": 0.8087887167930603,
70
+ "learning_rate": 5.0561797752809e-06,
71
+ "loss": 0.6562,
72
+ "step": 90
73
+ },
74
+ {
75
+ "epoch": 0.03380948356013862,
76
+ "grad_norm": 0.4273965656757355,
77
+ "learning_rate": 5.617977528089889e-06,
78
+ "loss": 0.5653,
79
+ "step": 100
80
+ },
81
+ {
82
+ "epoch": 0.03719043191615248,
83
+ "grad_norm": 0.3742749094963074,
84
+ "learning_rate": 6.179775280898876e-06,
85
+ "loss": 0.4352,
86
+ "step": 110
87
+ },
88
+ {
89
+ "epoch": 0.04057138027216634,
90
+ "grad_norm": 0.3029237985610962,
91
+ "learning_rate": 6.741573033707865e-06,
92
+ "loss": 0.534,
93
+ "step": 120
94
+ },
95
+ {
96
+ "epoch": 0.0439523286281802,
97
+ "grad_norm": 0.4938913583755493,
98
+ "learning_rate": 7.303370786516854e-06,
99
+ "loss": 0.4009,
100
+ "step": 130
101
+ },
102
+ {
103
+ "epoch": 0.04733327698419407,
104
+ "grad_norm": 0.3453095853328705,
105
+ "learning_rate": 7.865168539325843e-06,
106
+ "loss": 0.6226,
107
+ "step": 140
108
+ },
109
+ {
110
+ "epoch": 0.050714225340207926,
111
+ "grad_norm": 0.38411998748779297,
112
+ "learning_rate": 8.426966292134832e-06,
113
+ "loss": 0.3683,
114
+ "step": 150
115
+ },
116
+ {
117
+ "epoch": 0.05409517369622179,
118
+ "grad_norm": 0.34049779176712036,
119
+ "learning_rate": 8.988764044943822e-06,
120
+ "loss": 0.3729,
121
+ "step": 160
122
+ },
123
+ {
124
+ "epoch": 0.05747612205223565,
125
+ "grad_norm": 0.38956230878829956,
126
+ "learning_rate": 9.55056179775281e-06,
127
+ "loss": 0.3599,
128
+ "step": 170
129
+ },
130
+ {
131
+ "epoch": 0.06085707040824952,
132
+ "grad_norm": 0.3759077191352844,
133
+ "learning_rate": 9.99999700027518e-06,
134
+ "loss": 0.3587,
135
+ "step": 180
136
+ },
137
+ {
138
+ "epoch": 0.06423801876426338,
139
+ "grad_norm": 0.3305714726448059,
140
+ "learning_rate": 9.999892010284378e-06,
141
+ "loss": 0.3047,
142
+ "step": 190
143
+ },
144
+ {
145
+ "epoch": 0.06761896712027723,
146
+ "grad_norm": 0.3181019425392151,
147
+ "learning_rate": 9.999637037651849e-06,
148
+ "loss": 0.5866,
149
+ "step": 200
150
+ },
151
+ {
152
+ "epoch": 0.0709999154762911,
153
+ "grad_norm": 0.48781952261924744,
154
+ "learning_rate": 9.99923209002605e-06,
155
+ "loss": 0.3164,
156
+ "step": 210
157
+ },
158
+ {
159
+ "epoch": 0.07438086383230497,
160
+ "grad_norm": 0.4033997058868408,
161
+ "learning_rate": 9.99867717955427e-06,
162
+ "loss": 0.3258,
163
+ "step": 220
164
+ },
165
+ {
166
+ "epoch": 0.07776181218831882,
167
+ "grad_norm": 0.34120380878448486,
168
+ "learning_rate": 9.997972322882256e-06,
169
+ "loss": 0.4011,
170
+ "step": 230
171
+ },
172
+ {
173
+ "epoch": 0.08114276054433268,
174
+ "grad_norm": 0.2907439172267914,
175
+ "learning_rate": 9.997117541153717e-06,
176
+ "loss": 0.3496,
177
+ "step": 240
178
+ },
179
+ {
180
+ "epoch": 0.08452370890034655,
181
+ "grad_norm": 0.2975599467754364,
182
+ "learning_rate": 9.996112860009689e-06,
183
+ "loss": 0.3198,
184
+ "step": 250
185
+ },
186
+ {
187
+ "epoch": 0.0879046572563604,
188
+ "grad_norm": 0.406310111284256,
189
+ "learning_rate": 9.994958309587771e-06,
190
+ "loss": 0.3854,
191
+ "step": 260
192
+ },
193
+ {
194
+ "epoch": 0.09128560561237427,
195
+ "grad_norm": 0.45777568221092224,
196
+ "learning_rate": 9.993653924521214e-06,
197
+ "loss": 0.3544,
198
+ "step": 270
199
+ },
200
+ {
201
+ "epoch": 0.09466655396838813,
202
+ "grad_norm": 0.34425586462020874,
203
+ "learning_rate": 9.992199743937888e-06,
204
+ "loss": 0.3482,
205
+ "step": 280
206
+ },
207
+ {
208
+ "epoch": 0.098047502324402,
209
+ "grad_norm": 0.3571552336215973,
210
+ "learning_rate": 9.990595811459109e-06,
211
+ "loss": 0.3513,
212
+ "step": 290
213
+ },
214
+ {
215
+ "epoch": 0.10142845068041585,
216
+ "grad_norm": 0.40927043557167053,
217
+ "learning_rate": 9.988842175198314e-06,
218
+ "loss": 0.4002,
219
+ "step": 300
220
+ },
221
+ {
222
+ "epoch": 0.10480939903642972,
223
+ "grad_norm": 0.47294339537620544,
224
+ "learning_rate": 9.986938887759643e-06,
225
+ "loss": 0.4501,
226
+ "step": 310
227
+ },
228
+ {
229
+ "epoch": 0.10819034739244358,
230
+ "grad_norm": 0.4626659154891968,
231
+ "learning_rate": 9.984886006236344e-06,
232
+ "loss": 0.3541,
233
+ "step": 320
234
+ },
235
+ {
236
+ "epoch": 0.11157129574845744,
237
+ "grad_norm": 0.245002880692482,
238
+ "learning_rate": 9.982683592209069e-06,
239
+ "loss": 0.3402,
240
+ "step": 330
241
+ },
242
+ {
243
+ "epoch": 0.1149522441044713,
244
+ "grad_norm": 0.31318923830986023,
245
+ "learning_rate": 9.980331711744013e-06,
246
+ "loss": 0.379,
247
+ "step": 340
248
+ },
249
+ {
250
+ "epoch": 0.11833319246048517,
251
+ "grad_norm": 0.422484815120697,
252
+ "learning_rate": 9.977830435390954e-06,
253
+ "loss": 0.4657,
254
+ "step": 350
255
+ },
256
+ {
257
+ "epoch": 0.12171414081649903,
258
+ "grad_norm": 0.34094515442848206,
259
+ "learning_rate": 9.975179838181117e-06,
260
+ "loss": 0.4673,
261
+ "step": 360
262
+ },
263
+ {
264
+ "epoch": 0.12509508917251289,
265
+ "grad_norm": 0.37398943305015564,
266
+ "learning_rate": 9.972379999624935e-06,
267
+ "loss": 0.3145,
268
+ "step": 370
269
+ },
270
+ {
271
+ "epoch": 0.12847603752852677,
272
+ "grad_norm": 0.44416406750679016,
273
+ "learning_rate": 9.96943100370966e-06,
274
+ "loss": 0.3709,
275
+ "step": 380
276
+ },
277
+ {
278
+ "epoch": 0.13185698588454062,
279
+ "grad_norm": 0.4044680893421173,
280
+ "learning_rate": 9.966332938896839e-06,
281
+ "loss": 0.3761,
282
+ "step": 390
283
+ },
284
+ {
285
+ "epoch": 0.13523793424055447,
286
+ "grad_norm": 0.33949482440948486,
287
+ "learning_rate": 9.96308589811967e-06,
288
+ "loss": 0.3951,
289
+ "step": 400
290
+ },
291
+ {
292
+ "epoch": 0.13861888259656835,
293
+ "grad_norm": 0.4311394691467285,
294
+ "learning_rate": 9.959689978780207e-06,
295
+ "loss": 0.349,
296
+ "step": 410
297
+ },
298
+ {
299
+ "epoch": 0.1419998309525822,
300
+ "grad_norm": 0.37392905354499817,
301
+ "learning_rate": 9.95614528274644e-06,
302
+ "loss": 0.3283,
303
+ "step": 420
304
+ },
305
+ {
306
+ "epoch": 0.14538077930859605,
307
+ "grad_norm": 0.29748332500457764,
308
+ "learning_rate": 9.952451916349242e-06,
309
+ "loss": 0.3246,
310
+ "step": 430
311
+ },
312
+ {
313
+ "epoch": 0.14876172766460993,
314
+ "grad_norm": 0.3279096782207489,
315
+ "learning_rate": 9.948609990379175e-06,
316
+ "loss": 0.2735,
317
+ "step": 440
318
+ },
319
+ {
320
+ "epoch": 0.15214267602062378,
321
+ "grad_norm": 0.5759775042533875,
322
+ "learning_rate": 9.94461962008317e-06,
323
+ "loss": 0.3329,
324
+ "step": 450
325
+ },
326
+ {
327
+ "epoch": 0.15552362437663764,
328
+ "grad_norm": 0.3532526195049286,
329
+ "learning_rate": 9.940480925161065e-06,
330
+ "loss": 0.3502,
331
+ "step": 460
332
+ },
333
+ {
334
+ "epoch": 0.15890457273265152,
335
+ "grad_norm": 0.5062022805213928,
336
+ "learning_rate": 9.936194029762026e-06,
337
+ "loss": 0.3122,
338
+ "step": 470
339
+ },
340
+ {
341
+ "epoch": 0.16228552108866537,
342
+ "grad_norm": 0.39043861627578735,
343
+ "learning_rate": 9.931759062480804e-06,
344
+ "loss": 0.2812,
345
+ "step": 480
346
+ },
347
+ {
348
+ "epoch": 0.16566646944467922,
349
+ "grad_norm": 0.29016536474227905,
350
+ "learning_rate": 9.9271761563539e-06,
351
+ "loss": 0.2676,
352
+ "step": 490
353
+ },
354
+ {
355
+ "epoch": 0.1690474178006931,
356
+ "grad_norm": 0.33283042907714844,
357
+ "learning_rate": 9.92244544885555e-06,
358
+ "loss": 0.355,
359
+ "step": 500
360
+ },
361
+ {
362
+ "epoch": 0.17242836615670695,
363
+ "grad_norm": 0.3821840286254883,
364
+ "learning_rate": 9.917567081893628e-06,
365
+ "loss": 0.3255,
366
+ "step": 510
367
+ },
368
+ {
369
+ "epoch": 0.1758093145127208,
370
+ "grad_norm": 0.3762454688549042,
371
+ "learning_rate": 9.912541201805362e-06,
372
+ "loss": 0.3541,
373
+ "step": 520
374
+ },
375
+ {
376
+ "epoch": 0.17919026286873468,
377
+ "grad_norm": 0.360609233379364,
378
+ "learning_rate": 9.907367959352964e-06,
379
+ "loss": 0.3662,
380
+ "step": 530
381
+ },
382
+ {
383
+ "epoch": 0.18257121122474854,
384
+ "grad_norm": 0.3412153124809265,
385
+ "learning_rate": 9.902047509719101e-06,
386
+ "loss": 0.3337,
387
+ "step": 540
388
+ },
389
+ {
390
+ "epoch": 0.18595215958076242,
391
+ "grad_norm": 0.34308061003685,
392
+ "learning_rate": 9.896580012502238e-06,
393
+ "loss": 0.3648,
394
+ "step": 550
395
+ },
396
+ {
397
+ "epoch": 0.18933310793677627,
398
+ "grad_norm": 0.37141358852386475,
399
+ "learning_rate": 9.890965631711851e-06,
400
+ "loss": 0.3653,
401
+ "step": 560
402
+ },
403
+ {
404
+ "epoch": 0.19271405629279012,
405
+ "grad_norm": 0.2776310443878174,
406
+ "learning_rate": 9.88520453576351e-06,
407
+ "loss": 0.3575,
408
+ "step": 570
409
+ },
410
+ {
411
+ "epoch": 0.196095004648804,
412
+ "grad_norm": 0.39839184284210205,
413
+ "learning_rate": 9.879296897473825e-06,
414
+ "loss": 0.316,
415
+ "step": 580
416
+ },
417
+ {
418
+ "epoch": 0.19947595300481785,
419
+ "grad_norm": 0.4094005227088928,
420
+ "learning_rate": 9.873242894055265e-06,
421
+ "loss": 0.3241,
422
+ "step": 590
423
+ },
424
+ {
425
+ "epoch": 0.2028569013608317,
426
+ "grad_norm": 0.370228111743927,
427
+ "learning_rate": 9.867042707110837e-06,
428
+ "loss": 0.365,
429
+ "step": 600
430
+ },
431
+ {
432
+ "epoch": 0.20623784971684558,
433
+ "grad_norm": 0.41052767634391785,
434
+ "learning_rate": 9.860696522628638e-06,
435
+ "loss": 0.2937,
436
+ "step": 610
437
+ },
438
+ {
439
+ "epoch": 0.20961879807285944,
440
+ "grad_norm": 0.36997199058532715,
441
+ "learning_rate": 9.854204530976287e-06,
442
+ "loss": 0.5119,
443
+ "step": 620
444
+ },
445
+ {
446
+ "epoch": 0.2129997464288733,
447
+ "grad_norm": 0.43295958638191223,
448
+ "learning_rate": 9.847566926895199e-06,
449
+ "loss": 0.2889,
450
+ "step": 630
451
+ },
452
+ {
453
+ "epoch": 0.21638069478488717,
454
+ "grad_norm": 0.26491090655326843,
455
+ "learning_rate": 9.840783909494753e-06,
456
+ "loss": 0.3029,
457
+ "step": 640
458
+ },
459
+ {
460
+ "epoch": 0.21976164314090102,
461
+ "grad_norm": 0.36459994316101074,
462
+ "learning_rate": 9.833855682246319e-06,
463
+ "loss": 0.3101,
464
+ "step": 650
465
+ },
466
+ {
467
+ "epoch": 0.22314259149691487,
468
+ "grad_norm": 0.36685076355934143,
469
+ "learning_rate": 9.82678245297715e-06,
470
+ "loss": 0.3505,
471
+ "step": 660
472
+ },
473
+ {
474
+ "epoch": 0.22652353985292875,
475
+ "grad_norm": 0.37792932987213135,
476
+ "learning_rate": 9.81956443386415e-06,
477
+ "loss": 0.307,
478
+ "step": 670
479
+ },
480
+ {
481
+ "epoch": 0.2299044882089426,
482
+ "grad_norm": 0.9053909182548523,
483
+ "learning_rate": 9.81220184142751e-06,
484
+ "loss": 0.2987,
485
+ "step": 680
486
+ },
487
+ {
488
+ "epoch": 0.23328543656495648,
489
+ "grad_norm": 0.3331694006919861,
490
+ "learning_rate": 9.804694896524215e-06,
491
+ "loss": 0.2669,
492
+ "step": 690
493
+ },
494
+ {
495
+ "epoch": 0.23666638492097034,
496
+ "grad_norm": 0.38122400641441345,
497
+ "learning_rate": 9.797043824341413e-06,
498
+ "loss": 0.3466,
499
+ "step": 700
500
+ },
501
+ {
502
+ "epoch": 0.2400473332769842,
503
+ "grad_norm": 0.2942110300064087,
504
+ "learning_rate": 9.789248854389663e-06,
505
+ "loss": 0.3568,
506
+ "step": 710
507
+ },
508
+ {
509
+ "epoch": 0.24342828163299807,
510
+ "grad_norm": 0.3140552341938019,
511
+ "learning_rate": 9.781310220496056e-06,
512
+ "loss": 0.2744,
513
+ "step": 720
514
+ },
515
+ {
516
+ "epoch": 0.24680922998901192,
517
+ "grad_norm": 0.43632519245147705,
518
+ "learning_rate": 9.773228160797187e-06,
519
+ "loss": 0.2813,
520
+ "step": 730
521
+ },
522
+ {
523
+ "epoch": 0.25019017834502577,
524
+ "grad_norm": 0.3060181140899658,
525
+ "learning_rate": 9.765002917732029e-06,
526
+ "loss": 0.4297,
527
+ "step": 740
528
+ },
529
+ {
530
+ "epoch": 0.25357112670103965,
531
+ "grad_norm": 0.35231977701187134,
532
+ "learning_rate": 9.756634738034646e-06,
533
+ "loss": 0.3118,
534
+ "step": 750
535
+ },
536
+ {
537
+ "epoch": 0.25695207505705353,
538
+ "grad_norm": 0.29665321111679077,
539
+ "learning_rate": 9.748123872726801e-06,
540
+ "loss": 0.2527,
541
+ "step": 760
542
+ },
543
+ {
544
+ "epoch": 0.26033302341306735,
545
+ "grad_norm": 0.34136590361595154,
546
+ "learning_rate": 9.739470577110417e-06,
547
+ "loss": 0.2886,
548
+ "step": 770
549
+ },
550
+ {
551
+ "epoch": 0.26371397176908123,
552
+ "grad_norm": 0.3068552613258362,
553
+ "learning_rate": 9.730675110759929e-06,
554
+ "loss": 0.2795,
555
+ "step": 780
556
+ },
557
+ {
558
+ "epoch": 0.2670949201250951,
559
+ "grad_norm": 0.5336065888404846,
560
+ "learning_rate": 9.721737737514492e-06,
561
+ "loss": 0.2501,
562
+ "step": 790
563
+ },
564
+ {
565
+ "epoch": 0.27047586848110894,
566
+ "grad_norm": 0.33175092935562134,
567
+ "learning_rate": 9.712658725470066e-06,
568
+ "loss": 0.3423,
569
+ "step": 800
570
+ },
571
+ {
572
+ "epoch": 0.2738568168371228,
573
+ "grad_norm": 0.3154774308204651,
574
+ "learning_rate": 9.703438346971373e-06,
575
+ "loss": 0.2791,
576
+ "step": 810
577
+ },
578
+ {
579
+ "epoch": 0.2772377651931367,
580
+ "grad_norm": 0.3140163719654083,
581
+ "learning_rate": 9.694076878603736e-06,
582
+ "loss": 0.2966,
583
+ "step": 820
584
+ },
585
+ {
586
+ "epoch": 0.2806187135491505,
587
+ "grad_norm": 0.45935869216918945,
588
+ "learning_rate": 9.684574601184766e-06,
589
+ "loss": 0.2928,
590
+ "step": 830
591
+ },
592
+ {
593
+ "epoch": 0.2839996619051644,
594
+ "grad_norm": 0.4701893925666809,
595
+ "learning_rate": 9.674931799755958e-06,
596
+ "loss": 0.3848,
597
+ "step": 840
598
+ },
599
+ {
600
+ "epoch": 0.2873806102611783,
601
+ "grad_norm": 0.2994408905506134,
602
+ "learning_rate": 9.665148763574123e-06,
603
+ "loss": 0.3153,
604
+ "step": 850
605
+ },
606
+ {
607
+ "epoch": 0.2907615586171921,
608
+ "grad_norm": 0.39874333143234253,
609
+ "learning_rate": 9.655225786102724e-06,
610
+ "loss": 0.2409,
611
+ "step": 860
612
+ },
613
+ {
614
+ "epoch": 0.294142506973206,
615
+ "grad_norm": 0.42347922921180725,
616
+ "learning_rate": 9.645163165003063e-06,
617
+ "loss": 0.3434,
618
+ "step": 870
619
+ },
620
+ {
621
+ "epoch": 0.29752345532921987,
622
+ "grad_norm": 0.47837963700294495,
623
+ "learning_rate": 9.634961202125355e-06,
624
+ "loss": 0.3309,
625
+ "step": 880
626
+ },
627
+ {
628
+ "epoch": 0.3009044036852337,
629
+ "grad_norm": 0.3046788275241852,
630
+ "learning_rate": 9.624620203499683e-06,
631
+ "loss": 0.3413,
632
+ "step": 890
633
+ },
634
+ {
635
+ "epoch": 0.30428535204124757,
636
+ "grad_norm": 0.30118611454963684,
637
+ "learning_rate": 9.614140479326805e-06,
638
+ "loss": 0.2414,
639
+ "step": 900
640
+ },
641
+ {
642
+ "epoch": 0.30766630039726145,
643
+ "grad_norm": 0.3202741742134094,
644
+ "learning_rate": 9.603522343968852e-06,
645
+ "loss": 0.227,
646
+ "step": 910
647
+ },
648
+ {
649
+ "epoch": 0.3110472487532753,
650
+ "grad_norm": 0.36095985770225525,
651
+ "learning_rate": 9.592766115939902e-06,
652
+ "loss": 0.2662,
653
+ "step": 920
654
+ },
655
+ {
656
+ "epoch": 0.31442819710928915,
657
+ "grad_norm": 0.4588162899017334,
658
+ "learning_rate": 9.581872117896423e-06,
659
+ "loss": 0.338,
660
+ "step": 930
661
+ },
662
+ {
663
+ "epoch": 0.31780914546530303,
664
+ "grad_norm": 0.4730861485004425,
665
+ "learning_rate": 9.570840676627593e-06,
666
+ "loss": 0.3264,
667
+ "step": 940
668
+ },
669
+ {
670
+ "epoch": 0.32119009382131686,
671
+ "grad_norm": 0.27944037318229675,
672
+ "learning_rate": 9.5596721230455e-06,
673
+ "loss": 0.4616,
674
+ "step": 950
675
+ },
676
+ {
677
+ "epoch": 0.32457104217733074,
678
+ "grad_norm": 0.42319872975349426,
679
+ "learning_rate": 9.548366792175216e-06,
680
+ "loss": 0.3029,
681
+ "step": 960
682
+ },
683
+ {
684
+ "epoch": 0.3279519905333446,
685
+ "grad_norm": 0.34739917516708374,
686
+ "learning_rate": 9.536925023144742e-06,
687
+ "loss": 0.4111,
688
+ "step": 970
689
+ },
690
+ {
691
+ "epoch": 0.33133293888935844,
692
+ "grad_norm": 0.579397976398468,
693
+ "learning_rate": 9.52534715917484e-06,
694
+ "loss": 0.3451,
695
+ "step": 980
696
+ },
697
+ {
698
+ "epoch": 0.3347138872453723,
699
+ "grad_norm": 0.3507497310638428,
700
+ "learning_rate": 9.513633547568733e-06,
701
+ "loss": 0.3154,
702
+ "step": 990
703
+ },
704
+ {
705
+ "epoch": 0.3380948356013862,
706
+ "grad_norm": 0.42678698897361755,
707
+ "learning_rate": 9.501784539701697e-06,
708
+ "loss": 0.2993,
709
+ "step": 1000
710
+ },
711
+ {
712
+ "epoch": 0.3414757839574,
713
+ "grad_norm": 0.4824591875076294,
714
+ "learning_rate": 9.489800491010507e-06,
715
+ "loss": 0.3553,
716
+ "step": 1010
717
+ },
718
+ {
719
+ "epoch": 0.3448567323134139,
720
+ "grad_norm": 0.6306838393211365,
721
+ "learning_rate": 9.477681760982782e-06,
722
+ "loss": 0.3114,
723
+ "step": 1020
724
+ },
725
+ {
726
+ "epoch": 0.3482376806694278,
727
+ "grad_norm": 0.3566879630088806,
728
+ "learning_rate": 9.465428713146206e-06,
729
+ "loss": 0.263,
730
+ "step": 1030
731
+ },
732
+ {
733
+ "epoch": 0.3516186290254416,
734
+ "grad_norm": 0.26068150997161865,
735
+ "learning_rate": 9.45304171505761e-06,
736
+ "loss": 0.3227,
737
+ "step": 1040
738
+ },
739
+ {
740
+ "epoch": 0.3549995773814555,
741
+ "grad_norm": 0.33608248829841614,
742
+ "learning_rate": 9.440521138291962e-06,
743
+ "loss": 0.3224,
744
+ "step": 1050
745
+ },
746
+ {
747
+ "epoch": 0.35838052573746937,
748
+ "grad_norm": 0.3841913044452667,
749
+ "learning_rate": 9.427867358431209e-06,
750
+ "loss": 0.3401,
751
+ "step": 1060
752
+ },
753
+ {
754
+ "epoch": 0.36176147409348325,
755
+ "grad_norm": 0.3854820728302002,
756
+ "learning_rate": 9.415080755053014e-06,
757
+ "loss": 0.326,
758
+ "step": 1070
759
+ },
760
+ {
761
+ "epoch": 0.3651424224494971,
762
+ "grad_norm": 0.46526670455932617,
763
+ "learning_rate": 9.40216171171937e-06,
764
+ "loss": 0.264,
765
+ "step": 1080
766
+ },
767
+ {
768
+ "epoch": 0.36852337080551095,
769
+ "grad_norm": 0.40310919284820557,
770
+ "learning_rate": 9.389110615965102e-06,
771
+ "loss": 0.2192,
772
+ "step": 1090
773
+ },
774
+ {
775
+ "epoch": 0.37190431916152483,
776
+ "grad_norm": 0.3004761338233948,
777
+ "learning_rate": 9.375927859286224e-06,
778
+ "loss": 0.2863,
779
+ "step": 1100
780
+ },
781
+ {
782
+ "epoch": 0.37528526751753866,
783
+ "grad_norm": 0.3614630699157715,
784
+ "learning_rate": 9.362613837128215e-06,
785
+ "loss": 0.2849,
786
+ "step": 1110
787
+ },
788
+ {
789
+ "epoch": 0.37866621587355254,
790
+ "grad_norm": 0.33242204785346985,
791
+ "learning_rate": 9.34916894887414e-06,
792
+ "loss": 0.3034,
793
+ "step": 1120
794
+ },
795
+ {
796
+ "epoch": 0.3820471642295664,
797
+ "grad_norm": 0.2617999017238617,
798
+ "learning_rate": 9.335593597832686e-06,
799
+ "loss": 0.2769,
800
+ "step": 1130
801
+ },
802
+ {
803
+ "epoch": 0.38542811258558024,
804
+ "grad_norm": 0.2599431872367859,
805
+ "learning_rate": 9.321888191226047e-06,
806
+ "loss": 0.2641,
807
+ "step": 1140
808
+ },
809
+ {
810
+ "epoch": 0.3888090609415941,
811
+ "grad_norm": 0.3227129578590393,
812
+ "learning_rate": 9.308053140177722e-06,
813
+ "loss": 0.2852,
814
+ "step": 1150
815
+ },
816
+ {
817
+ "epoch": 0.392190009297608,
818
+ "grad_norm": 0.351832777261734,
819
+ "learning_rate": 9.294088859700173e-06,
820
+ "loss": 0.272,
821
+ "step": 1160
822
+ },
823
+ {
824
+ "epoch": 0.3955709576536218,
825
+ "grad_norm": 0.31848108768463135,
826
+ "learning_rate": 9.279995768682383e-06,
827
+ "loss": 0.2403,
828
+ "step": 1170
829
+ },
830
+ {
831
+ "epoch": 0.3989519060096357,
832
+ "grad_norm": 0.42877236008644104,
833
+ "learning_rate": 9.265774289877291e-06,
834
+ "loss": 0.2443,
835
+ "step": 1180
836
+ },
837
+ {
838
+ "epoch": 0.4023328543656496,
839
+ "grad_norm": 0.36654239892959595,
840
+ "learning_rate": 9.251424849889096e-06,
841
+ "loss": 0.2378,
842
+ "step": 1190
843
+ },
844
+ {
845
+ "epoch": 0.4057138027216634,
846
+ "grad_norm": 0.414516419172287,
847
+ "learning_rate": 9.236947879160483e-06,
848
+ "loss": 0.293,
849
+ "step": 1200
850
+ },
851
+ {
852
+ "epoch": 0.4090947510776773,
853
+ "grad_norm": 0.3989247679710388,
854
+ "learning_rate": 9.222343811959694e-06,
855
+ "loss": 0.2489,
856
+ "step": 1210
857
+ },
858
+ {
859
+ "epoch": 0.41247569943369117,
860
+ "grad_norm": 0.5263979434967041,
861
+ "learning_rate": 9.207613086367505e-06,
862
+ "loss": 0.309,
863
+ "step": 1220
864
+ },
865
+ {
866
+ "epoch": 0.415856647789705,
867
+ "grad_norm": 0.4520200788974762,
868
+ "learning_rate": 9.192756144264084e-06,
869
+ "loss": 0.2231,
870
+ "step": 1230
871
+ },
872
+ {
873
+ "epoch": 0.41923759614571887,
874
+ "grad_norm": 0.31832155585289,
875
+ "learning_rate": 9.177773431315748e-06,
876
+ "loss": 0.2561,
877
+ "step": 1240
878
+ },
879
+ {
880
+ "epoch": 0.42261854450173275,
881
+ "grad_norm": 0.4976191222667694,
882
+ "learning_rate": 9.162665396961573e-06,
883
+ "loss": 0.3474,
884
+ "step": 1250
885
+ },
886
+ {
887
+ "epoch": 0.4259994928577466,
888
+ "grad_norm": 0.4304470419883728,
889
+ "learning_rate": 9.147432494399929e-06,
890
+ "loss": 0.261,
891
+ "step": 1260
892
+ },
893
+ {
894
+ "epoch": 0.42938044121376046,
895
+ "grad_norm": 0.3462957739830017,
896
+ "learning_rate": 9.13207518057488e-06,
897
+ "loss": 0.2451,
898
+ "step": 1270
899
+ },
900
+ {
901
+ "epoch": 0.43276138956977434,
902
+ "grad_norm": 0.34633952379226685,
903
+ "learning_rate": 9.116593916162473e-06,
904
+ "loss": 0.3233,
905
+ "step": 1280
906
+ },
907
+ {
908
+ "epoch": 0.43614233792578816,
909
+ "grad_norm": 0.3529053032398224,
910
+ "learning_rate": 9.100989165556928e-06,
911
+ "loss": 0.2562,
912
+ "step": 1290
913
+ },
914
+ {
915
+ "epoch": 0.43952328628180204,
916
+ "grad_norm": 0.3165951073169708,
917
+ "learning_rate": 9.085261396856694e-06,
918
+ "loss": 0.3105,
919
+ "step": 1300
920
+ },
921
+ {
922
+ "epoch": 0.4429042346378159,
923
+ "grad_norm": 0.3625100553035736,
924
+ "learning_rate": 9.069411081850423e-06,
925
+ "loss": 0.2928,
926
+ "step": 1310
927
+ },
928
+ {
929
+ "epoch": 0.44628518299382974,
930
+ "grad_norm": 0.4361087381839752,
931
+ "learning_rate": 9.05343869600281e-06,
932
+ "loss": 0.3223,
933
+ "step": 1320
934
+ },
935
+ {
936
+ "epoch": 0.4496661313498436,
937
+ "grad_norm": 0.3407454490661621,
938
+ "learning_rate": 9.037344718440321e-06,
939
+ "loss": 0.2996,
940
+ "step": 1330
941
+ },
942
+ {
943
+ "epoch": 0.4530470797058575,
944
+ "grad_norm": 0.4694383144378662,
945
+ "learning_rate": 9.021129631936845e-06,
946
+ "loss": 0.3344,
947
+ "step": 1340
948
+ },
949
+ {
950
+ "epoch": 0.4564280280618714,
951
+ "grad_norm": 0.5816490054130554,
952
+ "learning_rate": 9.004793922899184e-06,
953
+ "loss": 0.3042,
954
+ "step": 1350
955
+ },
956
+ {
957
+ "epoch": 0.4598089764178852,
958
+ "grad_norm": 0.33371248841285706,
959
+ "learning_rate": 8.988338081352484e-06,
960
+ "loss": 0.4118,
961
+ "step": 1360
962
+ },
963
+ {
964
+ "epoch": 0.4631899247738991,
965
+ "grad_norm": 0.2931922674179077,
966
+ "learning_rate": 8.971762600925519e-06,
967
+ "loss": 0.2868,
968
+ "step": 1370
969
+ },
970
+ {
971
+ "epoch": 0.46657087312991297,
972
+ "grad_norm": 0.3763997256755829,
973
+ "learning_rate": 8.955067978835903e-06,
974
+ "loss": 0.3087,
975
+ "step": 1380
976
+ },
977
+ {
978
+ "epoch": 0.4699518214859268,
979
+ "grad_norm": 0.7063418626785278,
980
+ "learning_rate": 8.938254715875152e-06,
981
+ "loss": 0.2891,
982
+ "step": 1390
983
+ },
984
+ {
985
+ "epoch": 0.47333276984194067,
986
+ "grad_norm": 0.46078377962112427,
987
+ "learning_rate": 8.921323316393677e-06,
988
+ "loss": 0.2516,
989
+ "step": 1400
990
+ },
991
+ {
992
+ "epoch": 0.47671371819795455,
993
+ "grad_norm": 0.4854190945625305,
994
+ "learning_rate": 8.904274288285657e-06,
995
+ "loss": 0.2619,
996
+ "step": 1410
997
+ },
998
+ {
999
+ "epoch": 0.4800946665539684,
1000
+ "grad_norm": 0.3995590806007385,
1001
+ "learning_rate": 8.887108142973788e-06,
1002
+ "loss": 0.2596,
1003
+ "step": 1420
1004
+ },
1005
+ {
1006
+ "epoch": 0.48347561490998225,
1007
+ "grad_norm": 0.45142945647239685,
1008
+ "learning_rate": 8.869825395393958e-06,
1009
+ "loss": 0.277,
1010
+ "step": 1430
1011
+ },
1012
+ {
1013
+ "epoch": 0.48685656326599613,
1014
+ "grad_norm": 0.35526010394096375,
1015
+ "learning_rate": 8.852426563979793e-06,
1016
+ "loss": 0.2714,
1017
+ "step": 1440
1018
+ },
1019
+ {
1020
+ "epoch": 0.49023751162200996,
1021
+ "grad_norm": 0.4057399332523346,
1022
+ "learning_rate": 8.834912170647102e-06,
1023
+ "loss": 0.2526,
1024
+ "step": 1450
1025
+ },
1026
+ {
1027
+ "epoch": 0.49361845997802384,
1028
+ "grad_norm": 0.35347703099250793,
1029
+ "learning_rate": 8.81728274077823e-06,
1030
+ "loss": 0.3552,
1031
+ "step": 1460
1032
+ },
1033
+ {
1034
+ "epoch": 0.4969994083340377,
1035
+ "grad_norm": 0.37679535150527954,
1036
+ "learning_rate": 8.799538803206289e-06,
1037
+ "loss": 0.4208,
1038
+ "step": 1470
1039
+ },
1040
+ {
1041
+ "epoch": 0.5003803566900515,
1042
+ "grad_norm": 0.36378735303878784,
1043
+ "learning_rate": 8.781680890199306e-06,
1044
+ "loss": 0.2401,
1045
+ "step": 1480
1046
+ },
1047
+ {
1048
+ "epoch": 0.5037613050460654,
1049
+ "grad_norm": 0.4545590877532959,
1050
+ "learning_rate": 8.763709537444241e-06,
1051
+ "loss": 0.2768,
1052
+ "step": 1490
1053
+ },
1054
+ {
1055
+ "epoch": 0.5071422534020793,
1056
+ "grad_norm": 0.36166074872016907,
1057
+ "learning_rate": 8.745625284030932e-06,
1058
+ "loss": 0.3038,
1059
+ "step": 1500
1060
+ },
1061
+ {
1062
+ "epoch": 0.5105232017580932,
1063
+ "grad_norm": 0.27762094140052795,
1064
+ "learning_rate": 8.727428672435911e-06,
1065
+ "loss": 0.2748,
1066
+ "step": 1510
1067
+ },
1068
+ {
1069
+ "epoch": 0.5139041501141071,
1070
+ "grad_norm": 0.35694420337677,
1071
+ "learning_rate": 8.709120248506147e-06,
1072
+ "loss": 0.2813,
1073
+ "step": 1520
1074
+ },
1075
+ {
1076
+ "epoch": 0.5172850984701208,
1077
+ "grad_norm": 0.39979204535484314,
1078
+ "learning_rate": 8.690700561442658e-06,
1079
+ "loss": 0.2323,
1080
+ "step": 1530
1081
+ },
1082
+ {
1083
+ "epoch": 0.5206660468261347,
1084
+ "grad_norm": 0.3509082496166229,
1085
+ "learning_rate": 8.672170163784042e-06,
1086
+ "loss": 0.276,
1087
+ "step": 1540
1088
+ },
1089
+ {
1090
+ "epoch": 0.5240469951821486,
1091
+ "grad_norm": 0.3468097150325775,
1092
+ "learning_rate": 8.653529611389903e-06,
1093
+ "loss": 0.3483,
1094
+ "step": 1550
1095
+ },
1096
+ {
1097
+ "epoch": 0.5274279435381625,
1098
+ "grad_norm": 0.4471135437488556,
1099
+ "learning_rate": 8.634779463424177e-06,
1100
+ "loss": 0.2507,
1101
+ "step": 1560
1102
+ },
1103
+ {
1104
+ "epoch": 0.5308088918941763,
1105
+ "grad_norm": 0.34274283051490784,
1106
+ "learning_rate": 8.615920282338355e-06,
1107
+ "loss": 0.2879,
1108
+ "step": 1570
1109
+ },
1110
+ {
1111
+ "epoch": 0.5341898402501902,
1112
+ "grad_norm": 0.34896889328956604,
1113
+ "learning_rate": 8.596952633854617e-06,
1114
+ "loss": 0.3726,
1115
+ "step": 1580
1116
+ },
1117
+ {
1118
+ "epoch": 0.537570788606204,
1119
+ "grad_norm": 0.28104278445243835,
1120
+ "learning_rate": 8.577877086948859e-06,
1121
+ "loss": 0.35,
1122
+ "step": 1590
1123
+ },
1124
+ {
1125
+ "epoch": 0.5409517369622179,
1126
+ "grad_norm": 0.4057582914829254,
1127
+ "learning_rate": 8.558694213833618e-06,
1128
+ "loss": 0.3917,
1129
+ "step": 1600
1130
+ },
1131
+ {
1132
+ "epoch": 0.5443326853182318,
1133
+ "grad_norm": 0.5097754001617432,
1134
+ "learning_rate": 8.539404589940924e-06,
1135
+ "loss": 0.2214,
1136
+ "step": 1610
1137
+ },
1138
+ {
1139
+ "epoch": 0.5477136336742456,
1140
+ "grad_norm": 0.6435061097145081,
1141
+ "learning_rate": 8.52000879390502e-06,
1142
+ "loss": 0.276,
1143
+ "step": 1620
1144
+ },
1145
+ {
1146
+ "epoch": 0.5510945820302595,
1147
+ "grad_norm": 0.34463614225387573,
1148
+ "learning_rate": 8.50050740754502e-06,
1149
+ "loss": 0.3319,
1150
+ "step": 1630
1151
+ },
1152
+ {
1153
+ "epoch": 0.5544755303862734,
1154
+ "grad_norm": 0.4887869656085968,
1155
+ "learning_rate": 8.48090101584745e-06,
1156
+ "loss": 0.296,
1157
+ "step": 1640
1158
+ },
1159
+ {
1160
+ "epoch": 0.5578564787422872,
1161
+ "grad_norm": 0.3224658668041229,
1162
+ "learning_rate": 8.461190206948691e-06,
1163
+ "loss": 0.2329,
1164
+ "step": 1650
1165
+ },
1166
+ {
1167
+ "epoch": 0.561237427098301,
1168
+ "grad_norm": 0.4434487819671631,
1169
+ "learning_rate": 8.441375572117356e-06,
1170
+ "loss": 0.2664,
1171
+ "step": 1660
1172
+ },
1173
+ {
1174
+ "epoch": 0.5646183754543149,
1175
+ "grad_norm": 0.401167631149292,
1176
+ "learning_rate": 8.421457705736535e-06,
1177
+ "loss": 0.2821,
1178
+ "step": 1670
1179
+ },
1180
+ {
1181
+ "epoch": 0.5679993238103288,
1182
+ "grad_norm": 0.3888782858848572,
1183
+ "learning_rate": 8.401437205285977e-06,
1184
+ "loss": 0.2849,
1185
+ "step": 1680
1186
+ },
1187
+ {
1188
+ "epoch": 0.5713802721663427,
1189
+ "grad_norm": 0.4406028985977173,
1190
+ "learning_rate": 8.38131467132416e-06,
1191
+ "loss": 0.2654,
1192
+ "step": 1690
1193
+ },
1194
+ {
1195
+ "epoch": 0.5747612205223566,
1196
+ "grad_norm": 0.4088178873062134,
1197
+ "learning_rate": 8.361090707470286e-06,
1198
+ "loss": 0.4272,
1199
+ "step": 1700
1200
+ },
1201
+ {
1202
+ "epoch": 0.5781421688783703,
1203
+ "grad_norm": 0.3391360938549042,
1204
+ "learning_rate": 8.340765920386157e-06,
1205
+ "loss": 0.3114,
1206
+ "step": 1710
1207
+ },
1208
+ {
1209
+ "epoch": 0.5815231172343842,
1210
+ "grad_norm": 0.4771663546562195,
1211
+ "learning_rate": 8.320340919757997e-06,
1212
+ "loss": 0.2634,
1213
+ "step": 1720
1214
+ },
1215
+ {
1216
+ "epoch": 0.5849040655903981,
1217
+ "grad_norm": 0.32509908080101013,
1218
+ "learning_rate": 8.299816318278146e-06,
1219
+ "loss": 0.3494,
1220
+ "step": 1730
1221
+ },
1222
+ {
1223
+ "epoch": 0.588285013946412,
1224
+ "grad_norm": 0.36725375056266785,
1225
+ "learning_rate": 8.279192731626695e-06,
1226
+ "loss": 0.2748,
1227
+ "step": 1740
1228
+ },
1229
+ {
1230
+ "epoch": 0.5916659623024259,
1231
+ "grad_norm": 0.37348148226737976,
1232
+ "learning_rate": 8.258470778453005e-06,
1233
+ "loss": 0.2902,
1234
+ "step": 1750
1235
+ },
1236
+ {
1237
+ "epoch": 0.5950469106584397,
1238
+ "grad_norm": 0.2995922565460205,
1239
+ "learning_rate": 8.237651080357159e-06,
1240
+ "loss": 0.3042,
1241
+ "step": 1760
1242
+ },
1243
+ {
1244
+ "epoch": 0.5984278590144535,
1245
+ "grad_norm": 0.49292677640914917,
1246
+ "learning_rate": 8.216734261871305e-06,
1247
+ "loss": 0.2483,
1248
+ "step": 1770
1249
+ },
1250
+ {
1251
+ "epoch": 0.6018088073704674,
1252
+ "grad_norm": 0.39917120337486267,
1253
+ "learning_rate": 8.195720950440937e-06,
1254
+ "loss": 0.3498,
1255
+ "step": 1780
1256
+ },
1257
+ {
1258
+ "epoch": 0.6051897557264813,
1259
+ "grad_norm": 0.311768114566803,
1260
+ "learning_rate": 8.174611776406063e-06,
1261
+ "loss": 0.2725,
1262
+ "step": 1790
1263
+ },
1264
+ {
1265
+ "epoch": 0.6085707040824951,
1266
+ "grad_norm": 0.4066561460494995,
1267
+ "learning_rate": 8.15340737298229e-06,
1268
+ "loss": 0.3503,
1269
+ "step": 1800
1270
+ },
1271
+ {
1272
+ "epoch": 0.611951652438509,
1273
+ "grad_norm": 0.46395525336265564,
1274
+ "learning_rate": 8.132108376241849e-06,
1275
+ "loss": 0.3733,
1276
+ "step": 1810
1277
+ },
1278
+ {
1279
+ "epoch": 0.6153326007945229,
1280
+ "grad_norm": 0.39167824387550354,
1281
+ "learning_rate": 8.110715425094497e-06,
1282
+ "loss": 0.2493,
1283
+ "step": 1820
1284
+ },
1285
+ {
1286
+ "epoch": 0.6187135491505368,
1287
+ "grad_norm": 0.45918649435043335,
1288
+ "learning_rate": 8.089229161268361e-06,
1289
+ "loss": 0.2733,
1290
+ "step": 1830
1291
+ },
1292
+ {
1293
+ "epoch": 0.6220944975065505,
1294
+ "grad_norm": 0.3420335650444031,
1295
+ "learning_rate": 8.067650229290683e-06,
1296
+ "loss": 0.4992,
1297
+ "step": 1840
1298
+ },
1299
+ {
1300
+ "epoch": 0.6254754458625644,
1301
+ "grad_norm": 0.4455278515815735,
1302
+ "learning_rate": 8.045979276468486e-06,
1303
+ "loss": 0.2783,
1304
+ "step": 1850
1305
+ },
1306
+ {
1307
+ "epoch": 0.6288563942185783,
1308
+ "grad_norm": 0.3862534165382385,
1309
+ "learning_rate": 8.024216952869162e-06,
1310
+ "loss": 0.2567,
1311
+ "step": 1860
1312
+ },
1313
+ {
1314
+ "epoch": 0.6322373425745922,
1315
+ "grad_norm": 0.3253244161605835,
1316
+ "learning_rate": 8.002363911300966e-06,
1317
+ "loss": 0.2369,
1318
+ "step": 1870
1319
+ },
1320
+ {
1321
+ "epoch": 0.6356182909306061,
1322
+ "grad_norm": 0.35825368762016296,
1323
+ "learning_rate": 7.980420807293441e-06,
1324
+ "loss": 0.2241,
1325
+ "step": 1880
1326
+ },
1327
+ {
1328
+ "epoch": 0.63899923928662,
1329
+ "grad_norm": 0.3519473671913147,
1330
+ "learning_rate": 7.958388299077739e-06,
1331
+ "loss": 0.2643,
1332
+ "step": 1890
1333
+ },
1334
+ {
1335
+ "epoch": 0.6423801876426337,
1336
+ "grad_norm": 0.3944985270500183,
1337
+ "learning_rate": 7.936267047566897e-06,
1338
+ "loss": 0.2648,
1339
+ "step": 1900
1340
+ },
1341
+ {
1342
+ "epoch": 0.6457611359986476,
1343
+ "grad_norm": 0.465436190366745,
1344
+ "learning_rate": 7.914057716335992e-06,
1345
+ "loss": 0.3066,
1346
+ "step": 1910
1347
+ },
1348
+ {
1349
+ "epoch": 0.6491420843546615,
1350
+ "grad_norm": 0.306046724319458,
1351
+ "learning_rate": 7.891760971602248e-06,
1352
+ "loss": 0.3117,
1353
+ "step": 1920
1354
+ },
1355
+ {
1356
+ "epoch": 0.6525230327106754,
1357
+ "grad_norm": 0.39538854360580444,
1358
+ "learning_rate": 7.869377482205042e-06,
1359
+ "loss": 0.3432,
1360
+ "step": 1930
1361
+ },
1362
+ {
1363
+ "epoch": 0.6559039810666892,
1364
+ "grad_norm": 0.38408809900283813,
1365
+ "learning_rate": 7.846907919585856e-06,
1366
+ "loss": 0.2402,
1367
+ "step": 1940
1368
+ },
1369
+ {
1370
+ "epoch": 0.6592849294227031,
1371
+ "grad_norm": 0.38543111085891724,
1372
+ "learning_rate": 7.824352957768114e-06,
1373
+ "loss": 0.2739,
1374
+ "step": 1950
1375
+ },
1376
+ {
1377
+ "epoch": 0.6626658777787169,
1378
+ "grad_norm": 0.3631535470485687,
1379
+ "learning_rate": 7.801713273336984e-06,
1380
+ "loss": 0.3052,
1381
+ "step": 1960
1382
+ },
1383
+ {
1384
+ "epoch": 0.6660468261347308,
1385
+ "grad_norm": 0.36276668310165405,
1386
+ "learning_rate": 7.778989545419068e-06,
1387
+ "loss": 0.3565,
1388
+ "step": 1970
1389
+ },
1390
+ {
1391
+ "epoch": 0.6694277744907446,
1392
+ "grad_norm": 0.41759395599365234,
1393
+ "learning_rate": 7.756182455662038e-06,
1394
+ "loss": 0.3569,
1395
+ "step": 1980
1396
+ },
1397
+ {
1398
+ "epoch": 0.6728087228467585,
1399
+ "grad_norm": 0.4174079895019531,
1400
+ "learning_rate": 7.733292688214182e-06,
1401
+ "loss": 0.2461,
1402
+ "step": 1990
1403
+ },
1404
+ {
1405
+ "epoch": 0.6761896712027724,
1406
+ "grad_norm": 0.444947749376297,
1407
+ "learning_rate": 7.71032092970389e-06,
1408
+ "loss": 0.2597,
1409
+ "step": 2000
1410
+ },
1411
+ {
1412
+ "epoch": 0.6795706195587863,
1413
+ "grad_norm": 0.3082455098628998,
1414
+ "learning_rate": 7.687267869219052e-06,
1415
+ "loss": 0.2464,
1416
+ "step": 2010
1417
+ },
1418
+ {
1419
+ "epoch": 0.6829515679148,
1420
+ "grad_norm": 0.341994971036911,
1421
+ "learning_rate": 7.66413419828638e-06,
1422
+ "loss": 0.2379,
1423
+ "step": 2020
1424
+ },
1425
+ {
1426
+ "epoch": 0.6863325162708139,
1427
+ "grad_norm": 0.3558953106403351,
1428
+ "learning_rate": 7.640920610850684e-06,
1429
+ "loss": 0.3254,
1430
+ "step": 2030
1431
+ },
1432
+ {
1433
+ "epoch": 0.6897134646268278,
1434
+ "grad_norm": 0.35990360379219055,
1435
+ "learning_rate": 7.617627803254033e-06,
1436
+ "loss": 0.2722,
1437
+ "step": 2040
1438
+ },
1439
+ {
1440
+ "epoch": 0.6930944129828417,
1441
+ "grad_norm": 0.42061761021614075,
1442
+ "learning_rate": 7.594256474214883e-06,
1443
+ "loss": 0.2683,
1444
+ "step": 2050
1445
+ },
1446
+ {
1447
+ "epoch": 0.6964753613388556,
1448
+ "grad_norm": 0.373369038105011,
1449
+ "learning_rate": 7.57080732480711e-06,
1450
+ "loss": 0.3098,
1451
+ "step": 2060
1452
+ },
1453
+ {
1454
+ "epoch": 0.6998563096948694,
1455
+ "grad_norm": 0.39239752292633057,
1456
+ "learning_rate": 7.547281058438978e-06,
1457
+ "loss": 0.2583,
1458
+ "step": 2070
1459
+ },
1460
+ {
1461
+ "epoch": 0.7032372580508832,
1462
+ "grad_norm": 0.3450939655303955,
1463
+ "learning_rate": 7.523678380832049e-06,
1464
+ "loss": 0.2145,
1465
+ "step": 2080
1466
+ },
1467
+ {
1468
+ "epoch": 0.7066182064068971,
1469
+ "grad_norm": 0.29388755559921265,
1470
+ "learning_rate": 7.500000000000001e-06,
1471
+ "loss": 0.2775,
1472
+ "step": 2090
1473
+ },
1474
+ {
1475
+ "epoch": 0.709999154762911,
1476
+ "grad_norm": 0.36790722608566284,
1477
+ "learning_rate": 7.4762466262273945e-06,
1478
+ "loss": 0.2835,
1479
+ "step": 2100
1480
+ },
1481
+ {
1482
+ "epoch": 0.7133801031189249,
1483
+ "grad_norm": 0.4223284125328064,
1484
+ "learning_rate": 7.452418972048372e-06,
1485
+ "loss": 0.2621,
1486
+ "step": 2110
1487
+ },
1488
+ {
1489
+ "epoch": 0.7167610514749387,
1490
+ "grad_norm": 0.347509503364563,
1491
+ "learning_rate": 7.428517752225274e-06,
1492
+ "loss": 0.2706,
1493
+ "step": 2120
1494
+ },
1495
+ {
1496
+ "epoch": 0.7201419998309526,
1497
+ "grad_norm": 0.4447636008262634,
1498
+ "learning_rate": 7.404543683727201e-06,
1499
+ "loss": 0.3291,
1500
+ "step": 2130
1501
+ },
1502
+ {
1503
+ "epoch": 0.7235229481869665,
1504
+ "grad_norm": 0.35595768690109253,
1505
+ "learning_rate": 7.380497485708512e-06,
1506
+ "loss": 0.2613,
1507
+ "step": 2140
1508
+ },
1509
+ {
1510
+ "epoch": 0.7269038965429803,
1511
+ "grad_norm": 0.454505056142807,
1512
+ "learning_rate": 7.356379879487248e-06,
1513
+ "loss": 0.3418,
1514
+ "step": 2150
1515
+ },
1516
+ {
1517
+ "epoch": 0.7302848448989941,
1518
+ "grad_norm": 0.3137355446815491,
1519
+ "learning_rate": 7.33219158852349e-06,
1520
+ "loss": 0.2603,
1521
+ "step": 2160
1522
+ },
1523
+ {
1524
+ "epoch": 0.733665793255008,
1525
+ "grad_norm": 0.3765118718147278,
1526
+ "learning_rate": 7.307933338397667e-06,
1527
+ "loss": 0.3882,
1528
+ "step": 2170
1529
+ },
1530
+ {
1531
+ "epoch": 0.7370467416110219,
1532
+ "grad_norm": 1.2923246622085571,
1533
+ "learning_rate": 7.283605856788783e-06,
1534
+ "loss": 0.405,
1535
+ "step": 2180
1536
+ },
1537
+ {
1538
+ "epoch": 0.7404276899670358,
1539
+ "grad_norm": 0.4038814306259155,
1540
+ "learning_rate": 7.259209873452587e-06,
1541
+ "loss": 0.2474,
1542
+ "step": 2190
1543
+ },
1544
+ {
1545
+ "epoch": 0.7438086383230497,
1546
+ "grad_norm": 0.5514910221099854,
1547
+ "learning_rate": 7.234746120199695e-06,
1548
+ "loss": 0.3724,
1549
+ "step": 2200
1550
+ },
1551
+ {
1552
+ "epoch": 0.7471895866790634,
1553
+ "grad_norm": 0.3006429970264435,
1554
+ "learning_rate": 7.2102153308736225e-06,
1555
+ "loss": 0.2474,
1556
+ "step": 2210
1557
+ },
1558
+ {
1559
+ "epoch": 0.7505705350350773,
1560
+ "grad_norm": 0.4331114888191223,
1561
+ "learning_rate": 7.185618241328778e-06,
1562
+ "loss": 0.4156,
1563
+ "step": 2220
1564
+ },
1565
+ {
1566
+ "epoch": 0.7539514833910912,
1567
+ "grad_norm": 0.3075551986694336,
1568
+ "learning_rate": 7.160955589408395e-06,
1569
+ "loss": 0.2333,
1570
+ "step": 2230
1571
+ },
1572
+ {
1573
+ "epoch": 0.7573324317471051,
1574
+ "grad_norm": 0.3743632137775421,
1575
+ "learning_rate": 7.136228114922386e-06,
1576
+ "loss": 0.3248,
1577
+ "step": 2240
1578
+ },
1579
+ {
1580
+ "epoch": 0.760713380103119,
1581
+ "grad_norm": 0.5533598065376282,
1582
+ "learning_rate": 7.111436559625162e-06,
1583
+ "loss": 0.2368,
1584
+ "step": 2250
1585
+ },
1586
+ {
1587
+ "epoch": 0.7640943284591328,
1588
+ "grad_norm": 0.2940044403076172,
1589
+ "learning_rate": 7.0865816671933765e-06,
1590
+ "loss": 0.1987,
1591
+ "step": 2260
1592
+ },
1593
+ {
1594
+ "epoch": 0.7674752768151466,
1595
+ "grad_norm": 0.4096822142601013,
1596
+ "learning_rate": 7.061664183203617e-06,
1597
+ "loss": 0.2475,
1598
+ "step": 2270
1599
+ },
1600
+ {
1601
+ "epoch": 0.7708562251711605,
1602
+ "grad_norm": 0.3552418351173401,
1603
+ "learning_rate": 7.036684855110042e-06,
1604
+ "loss": 0.2641,
1605
+ "step": 2280
1606
+ },
1607
+ {
1608
+ "epoch": 0.7742371735271744,
1609
+ "grad_norm": 0.39436766505241394,
1610
+ "learning_rate": 7.0116444322219575e-06,
1611
+ "loss": 0.2576,
1612
+ "step": 2290
1613
+ },
1614
+ {
1615
+ "epoch": 0.7776181218831882,
1616
+ "grad_norm": 0.347412109375,
1617
+ "learning_rate": 6.986543665681344e-06,
1618
+ "loss": 0.3466,
1619
+ "step": 2300
1620
+ },
1621
+ {
1622
+ "epoch": 0.7809990702392021,
1623
+ "grad_norm": 0.5763669610023499,
1624
+ "learning_rate": 6.961383308440315e-06,
1625
+ "loss": 0.2916,
1626
+ "step": 2310
1627
+ },
1628
+ {
1629
+ "epoch": 0.784380018595216,
1630
+ "grad_norm": 0.826852560043335,
1631
+ "learning_rate": 6.936164115238543e-06,
1632
+ "loss": 0.3686,
1633
+ "step": 2320
1634
+ },
1635
+ {
1636
+ "epoch": 0.7877609669512298,
1637
+ "grad_norm": 0.5102378129959106,
1638
+ "learning_rate": 6.910886842580612e-06,
1639
+ "loss": 0.4308,
1640
+ "step": 2330
1641
+ },
1642
+ {
1643
+ "epoch": 0.7911419153072436,
1644
+ "grad_norm": 0.2508701980113983,
1645
+ "learning_rate": 6.8855522487133205e-06,
1646
+ "loss": 0.2969,
1647
+ "step": 2340
1648
+ },
1649
+ {
1650
+ "epoch": 0.7945228636632575,
1651
+ "grad_norm": 0.32886287569999695,
1652
+ "learning_rate": 6.860161093602949e-06,
1653
+ "loss": 0.2362,
1654
+ "step": 2350
1655
+ },
1656
+ {
1657
+ "epoch": 0.7979038120192714,
1658
+ "grad_norm": 0.5323238968849182,
1659
+ "learning_rate": 6.8347141389124495e-06,
1660
+ "loss": 0.2844,
1661
+ "step": 2360
1662
+ },
1663
+ {
1664
+ "epoch": 0.8012847603752853,
1665
+ "grad_norm": 0.31463027000427246,
1666
+ "learning_rate": 6.809212147978605e-06,
1667
+ "loss": 0.2684,
1668
+ "step": 2370
1669
+ },
1670
+ {
1671
+ "epoch": 0.8046657087312992,
1672
+ "grad_norm": 0.3970004916191101,
1673
+ "learning_rate": 6.783655885789136e-06,
1674
+ "loss": 0.2578,
1675
+ "step": 2380
1676
+ },
1677
+ {
1678
+ "epoch": 0.8080466570873129,
1679
+ "grad_norm": 0.3603476881980896,
1680
+ "learning_rate": 6.758046118959739e-06,
1681
+ "loss": 0.2729,
1682
+ "step": 2390
1683
+ },
1684
+ {
1685
+ "epoch": 0.8114276054433268,
1686
+ "grad_norm": 0.5765681862831116,
1687
+ "learning_rate": 6.732383615711104e-06,
1688
+ "loss": 0.2794,
1689
+ "step": 2400
1690
+ },
1691
+ {
1692
+ "epoch": 0.8148085537993407,
1693
+ "grad_norm": 0.3763329088687897,
1694
+ "learning_rate": 6.706669145845863e-06,
1695
+ "loss": 0.223,
1696
+ "step": 2410
1697
+ },
1698
+ {
1699
+ "epoch": 0.8181895021553546,
1700
+ "grad_norm": 0.37157511711120605,
1701
+ "learning_rate": 6.680903480725502e-06,
1702
+ "loss": 0.2393,
1703
+ "step": 2420
1704
+ },
1705
+ {
1706
+ "epoch": 0.8215704505113685,
1707
+ "grad_norm": 0.3405328691005707,
1708
+ "learning_rate": 6.655087393247216e-06,
1709
+ "loss": 0.234,
1710
+ "step": 2430
1711
+ },
1712
+ {
1713
+ "epoch": 0.8249513988673823,
1714
+ "grad_norm": 0.4525500535964966,
1715
+ "learning_rate": 6.629221657820729e-06,
1716
+ "loss": 0.2927,
1717
+ "step": 2440
1718
+ },
1719
+ {
1720
+ "epoch": 0.8283323472233962,
1721
+ "grad_norm": 0.4360927641391754,
1722
+ "learning_rate": 6.603307050345069e-06,
1723
+ "loss": 0.3324,
1724
+ "step": 2450
1725
+ },
1726
+ {
1727
+ "epoch": 0.83171329557941,
1728
+ "grad_norm": 0.5077038407325745,
1729
+ "learning_rate": 6.577344348185279e-06,
1730
+ "loss": 0.3252,
1731
+ "step": 2460
1732
+ },
1733
+ {
1734
+ "epoch": 0.8350942439354239,
1735
+ "grad_norm": 0.39226365089416504,
1736
+ "learning_rate": 6.551334330149114e-06,
1737
+ "loss": 0.3157,
1738
+ "step": 2470
1739
+ },
1740
+ {
1741
+ "epoch": 0.8384751922914377,
1742
+ "grad_norm": 1.4676735401153564,
1743
+ "learning_rate": 6.5252777764636666e-06,
1744
+ "loss": 0.2549,
1745
+ "step": 2480
1746
+ },
1747
+ {
1748
+ "epoch": 0.8418561406474516,
1749
+ "grad_norm": 0.46370625495910645,
1750
+ "learning_rate": 6.4991754687519695e-06,
1751
+ "loss": 0.3264,
1752
+ "step": 2490
1753
+ },
1754
+ {
1755
+ "epoch": 0.8452370890034655,
1756
+ "grad_norm": 0.27719298005104065,
1757
+ "learning_rate": 6.4730281900095474e-06,
1758
+ "loss": 0.2934,
1759
+ "step": 2500
1760
+ },
1761
+ {
1762
+ "epoch": 0.8486180373594794,
1763
+ "grad_norm": 0.37209928035736084,
1764
+ "learning_rate": 6.4468367245809296e-06,
1765
+ "loss": 0.2309,
1766
+ "step": 2510
1767
+ },
1768
+ {
1769
+ "epoch": 0.8519989857154932,
1770
+ "grad_norm": 0.38479083776474,
1771
+ "learning_rate": 6.42060185813612e-06,
1772
+ "loss": 0.2103,
1773
+ "step": 2520
1774
+ },
1775
+ {
1776
+ "epoch": 0.855379934071507,
1777
+ "grad_norm": 0.4944304823875427,
1778
+ "learning_rate": 6.394324377647028e-06,
1779
+ "loss": 0.2441,
1780
+ "step": 2530
1781
+ },
1782
+ {
1783
+ "epoch": 0.8587608824275209,
1784
+ "grad_norm": 0.3810581862926483,
1785
+ "learning_rate": 6.368005071363868e-06,
1786
+ "loss": 0.2739,
1787
+ "step": 2540
1788
+ },
1789
+ {
1790
+ "epoch": 0.8621418307835348,
1791
+ "grad_norm": 0.5139167904853821,
1792
+ "learning_rate": 6.341644728791509e-06,
1793
+ "loss": 0.2262,
1794
+ "step": 2550
1795
+ },
1796
+ {
1797
+ "epoch": 0.8655227791395487,
1798
+ "grad_norm": 0.3846452534198761,
1799
+ "learning_rate": 6.315244140665793e-06,
1800
+ "loss": 0.2173,
1801
+ "step": 2560
1802
+ },
1803
+ {
1804
+ "epoch": 0.8689037274955626,
1805
+ "grad_norm": 0.4736317992210388,
1806
+ "learning_rate": 6.2888040989298136e-06,
1807
+ "loss": 0.2335,
1808
+ "step": 2570
1809
+ },
1810
+ {
1811
+ "epoch": 0.8722846758515763,
1812
+ "grad_norm": 0.4954328238964081,
1813
+ "learning_rate": 6.262325396710161e-06,
1814
+ "loss": 0.2519,
1815
+ "step": 2580
1816
+ },
1817
+ {
1818
+ "epoch": 0.8756656242075902,
1819
+ "grad_norm": 0.3682522475719452,
1820
+ "learning_rate": 6.235808828293135e-06,
1821
+ "loss": 0.3018,
1822
+ "step": 2590
1823
+ },
1824
+ {
1825
+ "epoch": 0.8790465725636041,
1826
+ "grad_norm": 0.3742380738258362,
1827
+ "learning_rate": 6.209255189100908e-06,
1828
+ "loss": 0.2223,
1829
+ "step": 2600
1830
+ },
1831
+ {
1832
+ "epoch": 0.882427520919618,
1833
+ "grad_norm": 0.42900314927101135,
1834
+ "learning_rate": 6.182665275667674e-06,
1835
+ "loss": 0.2139,
1836
+ "step": 2610
1837
+ },
1838
+ {
1839
+ "epoch": 0.8858084692756318,
1840
+ "grad_norm": 0.4303857982158661,
1841
+ "learning_rate": 6.1560398856157554e-06,
1842
+ "loss": 0.2387,
1843
+ "step": 2620
1844
+ },
1845
+ {
1846
+ "epoch": 0.8891894176316457,
1847
+ "grad_norm": 0.4591567814350128,
1848
+ "learning_rate": 6.1293798176316664e-06,
1849
+ "loss": 0.3582,
1850
+ "step": 2630
1851
+ },
1852
+ {
1853
+ "epoch": 0.8925703659876595,
1854
+ "grad_norm": 0.33053892850875854,
1855
+ "learning_rate": 6.1026858714421666e-06,
1856
+ "loss": 0.2582,
1857
+ "step": 2640
1858
+ },
1859
+ {
1860
+ "epoch": 0.8959513143436734,
1861
+ "grad_norm": 0.7042179703712463,
1862
+ "learning_rate": 6.075958847790262e-06,
1863
+ "loss": 0.2902,
1864
+ "step": 2650
1865
+ },
1866
+ {
1867
+ "epoch": 0.8993322626996872,
1868
+ "grad_norm": 0.4624393582344055,
1869
+ "learning_rate": 6.049199548411192e-06,
1870
+ "loss": 0.3323,
1871
+ "step": 2660
1872
+ },
1873
+ {
1874
+ "epoch": 0.9027132110557011,
1875
+ "grad_norm": 0.35087037086486816,
1876
+ "learning_rate": 6.022408776008376e-06,
1877
+ "loss": 0.2899,
1878
+ "step": 2670
1879
+ },
1880
+ {
1881
+ "epoch": 0.906094159411715,
1882
+ "grad_norm": 0.3533821105957031,
1883
+ "learning_rate": 5.995587334229334e-06,
1884
+ "loss": 0.2689,
1885
+ "step": 2680
1886
+ },
1887
+ {
1888
+ "epoch": 0.9094751077677289,
1889
+ "grad_norm": 0.475006103515625,
1890
+ "learning_rate": 5.968736027641584e-06,
1891
+ "loss": 0.2815,
1892
+ "step": 2690
1893
+ },
1894
+ {
1895
+ "epoch": 0.9128560561237428,
1896
+ "grad_norm": 0.46712133288383484,
1897
+ "learning_rate": 5.941855661708496e-06,
1898
+ "loss": 0.2794,
1899
+ "step": 2700
1900
+ },
1901
+ {
1902
+ "epoch": 0.9162370044797565,
1903
+ "grad_norm": 0.34045907855033875,
1904
+ "learning_rate": 5.914947042765149e-06,
1905
+ "loss": 0.2455,
1906
+ "step": 2710
1907
+ },
1908
+ {
1909
+ "epoch": 0.9196179528357704,
1910
+ "grad_norm": 0.32822176814079285,
1911
+ "learning_rate": 5.888010977994128e-06,
1912
+ "loss": 0.2994,
1913
+ "step": 2720
1914
+ },
1915
+ {
1916
+ "epoch": 0.9229989011917843,
1917
+ "grad_norm": 0.4409944415092468,
1918
+ "learning_rate": 5.86104827540131e-06,
1919
+ "loss": 0.3139,
1920
+ "step": 2730
1921
+ },
1922
+ {
1923
+ "epoch": 0.9263798495477982,
1924
+ "grad_norm": 0.5034750699996948,
1925
+ "learning_rate": 5.8340597437916395e-06,
1926
+ "loss": 0.2495,
1927
+ "step": 2740
1928
+ },
1929
+ {
1930
+ "epoch": 0.929760797903812,
1931
+ "grad_norm": 0.5041275024414062,
1932
+ "learning_rate": 5.807046192744854e-06,
1933
+ "loss": 0.2931,
1934
+ "step": 2750
1935
+ },
1936
+ {
1937
+ "epoch": 0.9331417462598259,
1938
+ "grad_norm": 0.5224503874778748,
1939
+ "learning_rate": 5.780008432591204e-06,
1940
+ "loss": 0.3078,
1941
+ "step": 2760
1942
+ },
1943
+ {
1944
+ "epoch": 0.9365226946158397,
1945
+ "grad_norm": 0.5387886166572571,
1946
+ "learning_rate": 5.752947274387147e-06,
1947
+ "loss": 0.2573,
1948
+ "step": 2770
1949
+ },
1950
+ {
1951
+ "epoch": 0.9399036429718536,
1952
+ "grad_norm": 0.3771427869796753,
1953
+ "learning_rate": 5.725863529891015e-06,
1954
+ "loss": 0.3963,
1955
+ "step": 2780
1956
+ },
1957
+ {
1958
+ "epoch": 0.9432845913278675,
1959
+ "grad_norm": 0.36682406067848206,
1960
+ "learning_rate": 5.698758011538663e-06,
1961
+ "loss": 0.2167,
1962
+ "step": 2790
1963
+ },
1964
+ {
1965
+ "epoch": 0.9466655396838813,
1966
+ "grad_norm": 0.4087739586830139,
1967
+ "learning_rate": 5.6716315324191e-06,
1968
+ "loss": 0.2649,
1969
+ "step": 2800
1970
+ },
1971
+ {
1972
+ "epoch": 0.9500464880398952,
1973
+ "grad_norm": 0.40281644463539124,
1974
+ "learning_rate": 5.644484906250104e-06,
1975
+ "loss": 0.3459,
1976
+ "step": 2810
1977
+ },
1978
+ {
1979
+ "epoch": 0.9534274363959091,
1980
+ "grad_norm": 0.4454013705253601,
1981
+ "learning_rate": 5.617318947353803e-06,
1982
+ "loss": 0.3482,
1983
+ "step": 2820
1984
+ },
1985
+ {
1986
+ "epoch": 0.9568083847519229,
1987
+ "grad_norm": 0.3103313446044922,
1988
+ "learning_rate": 5.59013447063225e-06,
1989
+ "loss": 0.222,
1990
+ "step": 2830
1991
+ },
1992
+ {
1993
+ "epoch": 0.9601893331079367,
1994
+ "grad_norm": 0.4796975553035736,
1995
+ "learning_rate": 5.56293229154299e-06,
1996
+ "loss": 0.2742,
1997
+ "step": 2840
1998
+ },
1999
+ {
2000
+ "epoch": 0.9635702814639506,
2001
+ "grad_norm": 0.36471953988075256,
2002
+ "learning_rate": 5.535713226074576e-06,
2003
+ "loss": 0.2351,
2004
+ "step": 2850
2005
+ },
2006
+ {
2007
+ "epoch": 0.9669512298199645,
2008
+ "grad_norm": 0.42112353444099426,
2009
+ "learning_rate": 5.508478090722116e-06,
2010
+ "loss": 0.2504,
2011
+ "step": 2860
2012
+ },
2013
+ {
2014
+ "epoch": 0.9703321781759784,
2015
+ "grad_norm": 0.37776198983192444,
2016
+ "learning_rate": 5.481227702462762e-06,
2017
+ "loss": 0.2709,
2018
+ "step": 2870
2019
+ },
2020
+ {
2021
+ "epoch": 0.9737131265319923,
2022
+ "grad_norm": 0.32465189695358276,
2023
+ "learning_rate": 5.453962878731215e-06,
2024
+ "loss": 0.2217,
2025
+ "step": 2880
2026
+ },
2027
+ {
2028
+ "epoch": 0.977094074888006,
2029
+ "grad_norm": 0.3829999268054962,
2030
+ "learning_rate": 5.426684437395196e-06,
2031
+ "loss": 0.2331,
2032
+ "step": 2890
2033
+ },
2034
+ {
2035
+ "epoch": 0.9804750232440199,
2036
+ "grad_norm": 0.3728957772254944,
2037
+ "learning_rate": 5.399393196730917e-06,
2038
+ "loss": 0.3023,
2039
+ "step": 2900
2040
+ },
2041
+ {
2042
+ "epoch": 0.9838559716000338,
2043
+ "grad_norm": 0.4294697344303131,
2044
+ "learning_rate": 5.3720899753985344e-06,
2045
+ "loss": 0.2508,
2046
+ "step": 2910
2047
+ },
2048
+ {
2049
+ "epoch": 0.9872369199560477,
2050
+ "grad_norm": 0.3293459117412567,
2051
+ "learning_rate": 5.3447755924175885e-06,
2052
+ "loss": 0.2554,
2053
+ "step": 2920
2054
+ },
2055
+ {
2056
+ "epoch": 0.9906178683120616,
2057
+ "grad_norm": 0.3430132269859314,
2058
+ "learning_rate": 5.31745086714244e-06,
2059
+ "loss": 0.2403,
2060
+ "step": 2930
2061
+ },
2062
+ {
2063
+ "epoch": 0.9939988166680754,
2064
+ "grad_norm": 0.49164849519729614,
2065
+ "learning_rate": 5.2901166192376866e-06,
2066
+ "loss": 0.2985,
2067
+ "step": 2940
2068
+ },
2069
+ {
2070
+ "epoch": 0.9973797650240892,
2071
+ "grad_norm": 0.38642555475234985,
2072
+ "learning_rate": 5.26277366865358e-06,
2073
+ "loss": 0.2368,
2074
+ "step": 2950
2075
+ },
2076
+ {
2077
+ "epoch": 1.000760713380103,
2078
+ "grad_norm": 0.5772361755371094,
2079
+ "learning_rate": 5.235422835601431e-06,
2080
+ "loss": 0.296,
2081
+ "step": 2960
2082
+ },
2083
+ {
2084
+ "epoch": 1.004141661736117,
2085
+ "grad_norm": 0.3899407386779785,
2086
+ "learning_rate": 5.208064940528994e-06,
2087
+ "loss": 0.2751,
2088
+ "step": 2970
2089
+ },
2090
+ {
2091
+ "epoch": 1.0075226100921308,
2092
+ "grad_norm": 0.42107394337654114,
2093
+ "learning_rate": 5.180700804095871e-06,
2094
+ "loss": 0.3036,
2095
+ "step": 2980
2096
+ },
2097
+ {
2098
+ "epoch": 1.0109035584481447,
2099
+ "grad_norm": 0.3945677578449249,
2100
+ "learning_rate": 5.153331247148883e-06,
2101
+ "loss": 0.2095,
2102
+ "step": 2990
2103
+ },
2104
+ {
2105
+ "epoch": 1.0142845068041586,
2106
+ "grad_norm": 0.49374309182167053,
2107
+ "learning_rate": 5.125957090697453e-06,
2108
+ "loss": 0.2513,
2109
+ "step": 3000
2110
+ },
2111
+ {
2112
+ "epoch": 1.0176654551601725,
2113
+ "grad_norm": 0.37047889828681946,
2114
+ "learning_rate": 5.0985791558889785e-06,
2115
+ "loss": 0.2591,
2116
+ "step": 3010
2117
+ },
2118
+ {
2119
+ "epoch": 1.0210464035161864,
2120
+ "grad_norm": 0.5999889969825745,
2121
+ "learning_rate": 5.071198263984193e-06,
2122
+ "loss": 0.2596,
2123
+ "step": 3020
2124
+ },
2125
+ {
2126
+ "epoch": 1.0244273518722002,
2127
+ "grad_norm": 0.39319536089897156,
2128
+ "learning_rate": 5.0438152363325355e-06,
2129
+ "loss": 0.2684,
2130
+ "step": 3030
2131
+ },
2132
+ {
2133
+ "epoch": 1.0278083002282141,
2134
+ "grad_norm": 0.3694501221179962,
2135
+ "learning_rate": 5.016430894347513e-06,
2136
+ "loss": 0.2833,
2137
+ "step": 3040
2138
+ },
2139
+ {
2140
+ "epoch": 1.0311892485842278,
2141
+ "grad_norm": 0.4295886754989624,
2142
+ "learning_rate": 4.989046059482055e-06,
2143
+ "loss": 0.3042,
2144
+ "step": 3050
2145
+ },
2146
+ {
2147
+ "epoch": 1.0345701969402417,
2148
+ "grad_norm": 0.33182159066200256,
2149
+ "learning_rate": 4.961661553203883e-06,
2150
+ "loss": 0.2393,
2151
+ "step": 3060
2152
+ },
2153
+ {
2154
+ "epoch": 1.0379511452962555,
2155
+ "grad_norm": 0.32710009813308716,
2156
+ "learning_rate": 4.934278196970857e-06,
2157
+ "loss": 0.2908,
2158
+ "step": 3070
2159
+ },
2160
+ {
2161
+ "epoch": 1.0413320936522694,
2162
+ "grad_norm": 0.45676812529563904,
2163
+ "learning_rate": 4.906896812206336e-06,
2164
+ "loss": 0.2811,
2165
+ "step": 3080
2166
+ },
2167
+ {
2168
+ "epoch": 1.0447130420082833,
2169
+ "grad_norm": 0.42075178027153015,
2170
+ "learning_rate": 4.879518220274546e-06,
2171
+ "loss": 0.2905,
2172
+ "step": 3090
2173
+ },
2174
+ {
2175
+ "epoch": 1.0480939903642972,
2176
+ "grad_norm": 0.5601667165756226,
2177
+ "learning_rate": 4.852143242455935e-06,
2178
+ "loss": 0.2797,
2179
+ "step": 3100
2180
+ },
2181
+ {
2182
+ "epoch": 1.051474938720311,
2183
+ "grad_norm": 0.4164302349090576,
2184
+ "learning_rate": 4.824772699922535e-06,
2185
+ "loss": 0.3133,
2186
+ "step": 3110
2187
+ },
2188
+ {
2189
+ "epoch": 1.054855887076325,
2190
+ "grad_norm": 0.6831467151641846,
2191
+ "learning_rate": 4.797407413713334e-06,
2192
+ "loss": 0.2528,
2193
+ "step": 3120
2194
+ },
2195
+ {
2196
+ "epoch": 1.0582368354323388,
2197
+ "grad_norm": 0.4091845750808716,
2198
+ "learning_rate": 4.770048204709648e-06,
2199
+ "loss": 0.2846,
2200
+ "step": 3130
2201
+ },
2202
+ {
2203
+ "epoch": 1.0616177837883527,
2204
+ "grad_norm": 0.43544697761535645,
2205
+ "learning_rate": 4.742695893610486e-06,
2206
+ "loss": 0.2762,
2207
+ "step": 3140
2208
+ },
2209
+ {
2210
+ "epoch": 1.0649987321443666,
2211
+ "grad_norm": 0.398639976978302,
2212
+ "learning_rate": 4.715351300907946e-06,
2213
+ "loss": 0.2237,
2214
+ "step": 3150
2215
+ },
2216
+ {
2217
+ "epoch": 1.0683796805003805,
2218
+ "grad_norm": 0.472295880317688,
2219
+ "learning_rate": 4.688015246862595e-06,
2220
+ "loss": 0.2515,
2221
+ "step": 3160
2222
+ },
2223
+ {
2224
+ "epoch": 1.0717606288563941,
2225
+ "grad_norm": 0.3498964011669159,
2226
+ "learning_rate": 4.660688551478861e-06,
2227
+ "loss": 0.2291,
2228
+ "step": 3170
2229
+ },
2230
+ {
2231
+ "epoch": 1.075141577212408,
2232
+ "grad_norm": 0.620032012462616,
2233
+ "learning_rate": 4.633372034480444e-06,
2234
+ "loss": 0.2899,
2235
+ "step": 3180
2236
+ },
2237
+ {
2238
+ "epoch": 1.0785225255684219,
2239
+ "grad_norm": 0.3728788197040558,
2240
+ "learning_rate": 4.606066515285719e-06,
2241
+ "loss": 0.2784,
2242
+ "step": 3190
2243
+ },
2244
+ {
2245
+ "epoch": 1.0819034739244358,
2246
+ "grad_norm": 0.6266978979110718,
2247
+ "learning_rate": 4.578772812983151e-06,
2248
+ "loss": 0.2504,
2249
+ "step": 3200
2250
+ },
2251
+ {
2252
+ "epoch": 1.0852844222804496,
2253
+ "grad_norm": 0.5173726081848145,
2254
+ "learning_rate": 4.551491746306744e-06,
2255
+ "loss": 0.2907,
2256
+ "step": 3210
2257
+ },
2258
+ {
2259
+ "epoch": 1.0886653706364635,
2260
+ "grad_norm": 0.4330489933490753,
2261
+ "learning_rate": 4.5242241336114595e-06,
2262
+ "loss": 0.2149,
2263
+ "step": 3220
2264
+ },
2265
+ {
2266
+ "epoch": 1.0920463189924774,
2267
+ "grad_norm": 0.3191027343273163,
2268
+ "learning_rate": 4.4969707928486815e-06,
2269
+ "loss": 0.2043,
2270
+ "step": 3230
2271
+ },
2272
+ {
2273
+ "epoch": 1.0954272673484913,
2274
+ "grad_norm": 0.414315402507782,
2275
+ "learning_rate": 4.469732541541675e-06,
2276
+ "loss": 0.293,
2277
+ "step": 3240
2278
+ },
2279
+ {
2280
+ "epoch": 1.0988082157045052,
2281
+ "grad_norm": 0.4167482852935791,
2282
+ "learning_rate": 4.442510196761068e-06,
2283
+ "loss": 0.3487,
2284
+ "step": 3250
2285
+ },
2286
+ {
2287
+ "epoch": 1.102189164060519,
2288
+ "grad_norm": 0.40039610862731934,
2289
+ "learning_rate": 4.4153045751003294e-06,
2290
+ "loss": 0.2494,
2291
+ "step": 3260
2292
+ },
2293
+ {
2294
+ "epoch": 1.105570112416533,
2295
+ "grad_norm": 0.5247258543968201,
2296
+ "learning_rate": 4.388116492651287e-06,
2297
+ "loss": 0.2506,
2298
+ "step": 3270
2299
+ },
2300
+ {
2301
+ "epoch": 1.1089510607725468,
2302
+ "grad_norm": 0.5273059606552124,
2303
+ "learning_rate": 4.3609467649796434e-06,
2304
+ "loss": 0.2887,
2305
+ "step": 3280
2306
+ },
2307
+ {
2308
+ "epoch": 1.1123320091285605,
2309
+ "grad_norm": 0.42885908484458923,
2310
+ "learning_rate": 4.333796207100505e-06,
2311
+ "loss": 0.3104,
2312
+ "step": 3290
2313
+ },
2314
+ {
2315
+ "epoch": 1.1157129574845743,
2316
+ "grad_norm": 0.4897991120815277,
2317
+ "learning_rate": 4.3066656334539435e-06,
2318
+ "loss": 0.2603,
2319
+ "step": 3300
2320
+ },
2321
+ {
2322
+ "epoch": 1.1190939058405882,
2323
+ "grad_norm": 0.5316048264503479,
2324
+ "learning_rate": 4.279555857880558e-06,
2325
+ "loss": 0.2276,
2326
+ "step": 3310
2327
+ },
2328
+ {
2329
+ "epoch": 1.122474854196602,
2330
+ "grad_norm": 0.4185865521430969,
2331
+ "learning_rate": 4.252467693597059e-06,
2332
+ "loss": 0.31,
2333
+ "step": 3320
2334
+ },
2335
+ {
2336
+ "epoch": 1.125855802552616,
2337
+ "grad_norm": 0.4467753767967224,
2338
+ "learning_rate": 4.22540195317189e-06,
2339
+ "loss": 0.3056,
2340
+ "step": 3330
2341
+ },
2342
+ {
2343
+ "epoch": 1.1292367509086298,
2344
+ "grad_norm": 0.4325476288795471,
2345
+ "learning_rate": 4.198359448500832e-06,
2346
+ "loss": 0.2376,
2347
+ "step": 3340
2348
+ },
2349
+ {
2350
+ "epoch": 1.1326176992646437,
2351
+ "grad_norm": 0.3991256356239319,
2352
+ "learning_rate": 4.171340990782664e-06,
2353
+ "loss": 0.2496,
2354
+ "step": 3350
2355
+ },
2356
+ {
2357
+ "epoch": 1.1359986476206576,
2358
+ "grad_norm": 0.5971009135246277,
2359
+ "learning_rate": 4.144347390494824e-06,
2360
+ "loss": 0.3391,
2361
+ "step": 3360
2362
+ },
2363
+ {
2364
+ "epoch": 1.1393795959766715,
2365
+ "grad_norm": 0.5420436263084412,
2366
+ "learning_rate": 4.1173794573691e-06,
2367
+ "loss": 0.2621,
2368
+ "step": 3370
2369
+ },
2370
+ {
2371
+ "epoch": 1.1427605443326854,
2372
+ "grad_norm": 0.4983433485031128,
2373
+ "learning_rate": 4.090438000367328e-06,
2374
+ "loss": 0.2745,
2375
+ "step": 3380
2376
+ },
2377
+ {
2378
+ "epoch": 1.1461414926886992,
2379
+ "grad_norm": 0.4821936786174774,
2380
+ "learning_rate": 4.0635238276571445e-06,
2381
+ "loss": 0.262,
2382
+ "step": 3390
2383
+ },
2384
+ {
2385
+ "epoch": 1.1495224410447131,
2386
+ "grad_norm": 0.45023828744888306,
2387
+ "learning_rate": 4.036637746587732e-06,
2388
+ "loss": 0.4338,
2389
+ "step": 3400
2390
+ },
2391
+ {
2392
+ "epoch": 1.152903389400727,
2393
+ "grad_norm": 0.355067640542984,
2394
+ "learning_rate": 4.009780563665601e-06,
2395
+ "loss": 0.2192,
2396
+ "step": 3410
2397
+ },
2398
+ {
2399
+ "epoch": 1.1562843377567407,
2400
+ "grad_norm": 0.41529661417007446,
2401
+ "learning_rate": 3.9829530845304005e-06,
2402
+ "loss": 0.2691,
2403
+ "step": 3420
2404
+ },
2405
+ {
2406
+ "epoch": 1.1596652861127545,
2407
+ "grad_norm": 0.45665085315704346,
2408
+ "learning_rate": 3.956156113930752e-06,
2409
+ "loss": 0.2126,
2410
+ "step": 3430
2411
+ },
2412
+ {
2413
+ "epoch": 1.1630462344687684,
2414
+ "grad_norm": 0.49977874755859375,
2415
+ "learning_rate": 3.9293904557001005e-06,
2416
+ "loss": 0.3299,
2417
+ "step": 3440
2418
+ },
2419
+ {
2420
+ "epoch": 1.1664271828247823,
2421
+ "grad_norm": 0.4577089548110962,
2422
+ "learning_rate": 3.902656912732616e-06,
2423
+ "loss": 0.21,
2424
+ "step": 3450
2425
+ },
2426
+ {
2427
+ "epoch": 1.1698081311807962,
2428
+ "grad_norm": 0.3317941725254059,
2429
+ "learning_rate": 3.875956286959096e-06,
2430
+ "loss": 0.1909,
2431
+ "step": 3460
2432
+ },
2433
+ {
2434
+ "epoch": 1.17318907953681,
2435
+ "grad_norm": 0.3392845094203949,
2436
+ "learning_rate": 3.849289379322919e-06,
2437
+ "loss": 0.2369,
2438
+ "step": 3470
2439
+ },
2440
+ {
2441
+ "epoch": 1.176570027892824,
2442
+ "grad_norm": 0.3934585750102997,
2443
+ "learning_rate": 3.82265698975601e-06,
2444
+ "loss": 0.2149,
2445
+ "step": 3480
2446
+ },
2447
+ {
2448
+ "epoch": 1.1799509762488378,
2449
+ "grad_norm": 0.4721519947052002,
2450
+ "learning_rate": 3.7960599171548572e-06,
2451
+ "loss": 0.2608,
2452
+ "step": 3490
2453
+ },
2454
+ {
2455
+ "epoch": 1.1833319246048517,
2456
+ "grad_norm": 0.4335407614707947,
2457
+ "learning_rate": 3.7694989593565295e-06,
2458
+ "loss": 0.447,
2459
+ "step": 3500
2460
+ },
2461
+ {
2462
+ "epoch": 1.1867128729608656,
2463
+ "grad_norm": 0.6267192363739014,
2464
+ "learning_rate": 3.742974913114759e-06,
2465
+ "loss": 0.2752,
2466
+ "step": 3510
2467
+ },
2468
+ {
2469
+ "epoch": 1.1900938213168795,
2470
+ "grad_norm": 0.42228975892066956,
2471
+ "learning_rate": 3.7164885740760375e-06,
2472
+ "loss": 0.2648,
2473
+ "step": 3520
2474
+ },
2475
+ {
2476
+ "epoch": 1.1934747696728933,
2477
+ "grad_norm": 0.44687971472740173,
2478
+ "learning_rate": 3.690040736755742e-06,
2479
+ "loss": 0.2308,
2480
+ "step": 3530
2481
+ },
2482
+ {
2483
+ "epoch": 1.1968557180289072,
2484
+ "grad_norm": 0.37984123826026917,
2485
+ "learning_rate": 3.6636321945143117e-06,
2486
+ "loss": 0.3304,
2487
+ "step": 3540
2488
+ },
2489
+ {
2490
+ "epoch": 1.2002366663849209,
2491
+ "grad_norm": 0.3920864164829254,
2492
+ "learning_rate": 3.6372637395334416e-06,
2493
+ "loss": 0.2834,
2494
+ "step": 3550
2495
+ },
2496
+ {
2497
+ "epoch": 1.2036176147409348,
2498
+ "grad_norm": 0.49319878220558167,
2499
+ "learning_rate": 3.6109361627923194e-06,
2500
+ "loss": 0.3353,
2501
+ "step": 3560
2502
+ },
2503
+ {
2504
+ "epoch": 1.2069985630969486,
2505
+ "grad_norm": 0.34764301776885986,
2506
+ "learning_rate": 3.5846502540439076e-06,
2507
+ "loss": 0.2821,
2508
+ "step": 3570
2509
+ },
2510
+ {
2511
+ "epoch": 1.2103795114529625,
2512
+ "grad_norm": 0.5755962133407593,
2513
+ "learning_rate": 3.5584068017912415e-06,
2514
+ "loss": 0.2331,
2515
+ "step": 3580
2516
+ },
2517
+ {
2518
+ "epoch": 1.2137604598089764,
2519
+ "grad_norm": 0.38585567474365234,
2520
+ "learning_rate": 3.5322065932637838e-06,
2521
+ "loss": 0.3051,
2522
+ "step": 3590
2523
+ },
2524
+ {
2525
+ "epoch": 1.2171414081649903,
2526
+ "grad_norm": 0.48157286643981934,
2527
+ "learning_rate": 3.5060504143938056e-06,
2528
+ "loss": 0.2099,
2529
+ "step": 3600
2530
+ },
2531
+ {
2532
+ "epoch": 1.2205223565210042,
2533
+ "grad_norm": 0.30920207500457764,
2534
+ "learning_rate": 3.479939049792817e-06,
2535
+ "loss": 0.2164,
2536
+ "step": 3610
2537
+ },
2538
+ {
2539
+ "epoch": 1.223903304877018,
2540
+ "grad_norm": 1.0118547677993774,
2541
+ "learning_rate": 3.4538732827280197e-06,
2542
+ "loss": 0.2664,
2543
+ "step": 3620
2544
+ },
2545
+ {
2546
+ "epoch": 1.227284253233032,
2547
+ "grad_norm": 0.334075927734375,
2548
+ "learning_rate": 3.427853895098822e-06,
2549
+ "loss": 0.2272,
2550
+ "step": 3630
2551
+ },
2552
+ {
2553
+ "epoch": 1.2306652015890458,
2554
+ "grad_norm": 0.4185827970504761,
2555
+ "learning_rate": 3.4018816674133814e-06,
2556
+ "loss": 0.3044,
2557
+ "step": 3640
2558
+ },
2559
+ {
2560
+ "epoch": 1.2340461499450597,
2561
+ "grad_norm": 0.4281156063079834,
2562
+ "learning_rate": 3.3759573787651877e-06,
2563
+ "loss": 0.2189,
2564
+ "step": 3650
2565
+ },
2566
+ {
2567
+ "epoch": 1.2374270983010733,
2568
+ "grad_norm": 0.5715534090995789,
2569
+ "learning_rate": 3.3500818068096976e-06,
2570
+ "loss": 0.2345,
2571
+ "step": 3660
2572
+ },
2573
+ {
2574
+ "epoch": 1.2408080466570874,
2575
+ "grad_norm": 0.5237227082252502,
2576
+ "learning_rate": 3.3242557277410015e-06,
2577
+ "loss": 0.3478,
2578
+ "step": 3670
2579
+ },
2580
+ {
2581
+ "epoch": 1.244188995013101,
2582
+ "grad_norm": 0.419845849275589,
2583
+ "learning_rate": 3.298479916268543e-06,
2584
+ "loss": 0.228,
2585
+ "step": 3680
2586
+ },
2587
+ {
2588
+ "epoch": 1.247569943369115,
2589
+ "grad_norm": 0.4372294247150421,
2590
+ "learning_rate": 3.2727551455938832e-06,
2591
+ "loss": 0.2565,
2592
+ "step": 3690
2593
+ },
2594
+ {
2595
+ "epoch": 1.2509508917251289,
2596
+ "grad_norm": 0.3739818036556244,
2597
+ "learning_rate": 3.2470821873875003e-06,
2598
+ "loss": 0.3229,
2599
+ "step": 3700
2600
+ },
2601
+ {
2602
+ "epoch": 1.2543318400811427,
2603
+ "grad_norm": 0.3473162353038788,
2604
+ "learning_rate": 3.221461811765647e-06,
2605
+ "loss": 0.2555,
2606
+ "step": 3710
2607
+ },
2608
+ {
2609
+ "epoch": 1.2577127884371566,
2610
+ "grad_norm": 0.40606626868247986,
2611
+ "learning_rate": 3.1958947872672436e-06,
2612
+ "loss": 0.256,
2613
+ "step": 3720
2614
+ },
2615
+ {
2616
+ "epoch": 1.2610937367931705,
2617
+ "grad_norm": 0.49288639426231384,
2618
+ "learning_rate": 3.1703818808308327e-06,
2619
+ "loss": 0.2597,
2620
+ "step": 3730
2621
+ },
2622
+ {
2623
+ "epoch": 1.2644746851491844,
2624
+ "grad_norm": 0.5764711499214172,
2625
+ "learning_rate": 3.1449238577715625e-06,
2626
+ "loss": 0.309,
2627
+ "step": 3740
2628
+ },
2629
+ {
2630
+ "epoch": 1.2678556335051983,
2631
+ "grad_norm": 0.33573490381240845,
2632
+ "learning_rate": 3.1195214817582354e-06,
2633
+ "loss": 0.2043,
2634
+ "step": 3750
2635
+ },
2636
+ {
2637
+ "epoch": 1.2712365818612121,
2638
+ "grad_norm": 0.45355305075645447,
2639
+ "learning_rate": 3.0941755147904027e-06,
2640
+ "loss": 0.2828,
2641
+ "step": 3760
2642
+ },
2643
+ {
2644
+ "epoch": 1.274617530217226,
2645
+ "grad_norm": 0.44018998742103577,
2646
+ "learning_rate": 3.0688867171755043e-06,
2647
+ "loss": 0.2124,
2648
+ "step": 3770
2649
+ },
2650
+ {
2651
+ "epoch": 1.27799847857324,
2652
+ "grad_norm": 0.5433792471885681,
2653
+ "learning_rate": 3.043655847506055e-06,
2654
+ "loss": 0.2551,
2655
+ "step": 3780
2656
+ },
2657
+ {
2658
+ "epoch": 1.2813794269292536,
2659
+ "grad_norm": 0.41077756881713867,
2660
+ "learning_rate": 3.0184836626369034e-06,
2661
+ "loss": 0.2429,
2662
+ "step": 3790
2663
+ },
2664
+ {
2665
+ "epoch": 1.2847603752852677,
2666
+ "grad_norm": 0.5044552683830261,
2667
+ "learning_rate": 2.9933709176625092e-06,
2668
+ "loss": 0.2428,
2669
+ "step": 3800
2670
+ },
2671
+ {
2672
+ "epoch": 1.2881413236412813,
2673
+ "grad_norm": 0.841780424118042,
2674
+ "learning_rate": 2.9683183658943106e-06,
2675
+ "loss": 0.2564,
2676
+ "step": 3810
2677
+ },
2678
+ {
2679
+ "epoch": 1.2915222719972952,
2680
+ "grad_norm": 0.3177238702774048,
2681
+ "learning_rate": 2.9433267588381198e-06,
2682
+ "loss": 0.3334,
2683
+ "step": 3820
2684
+ },
2685
+ {
2686
+ "epoch": 1.294903220353309,
2687
+ "grad_norm": 0.40576761960983276,
2688
+ "learning_rate": 2.918396846171577e-06,
2689
+ "loss": 0.269,
2690
+ "step": 3830
2691
+ },
2692
+ {
2693
+ "epoch": 1.298284168709323,
2694
+ "grad_norm": 0.4299604296684265,
2695
+ "learning_rate": 2.8935293757216643e-06,
2696
+ "loss": 0.2815,
2697
+ "step": 3840
2698
+ },
2699
+ {
2700
+ "epoch": 1.3016651170653368,
2701
+ "grad_norm": 0.47269025444984436,
2702
+ "learning_rate": 2.8687250934422774e-06,
2703
+ "loss": 0.4005,
2704
+ "step": 3850
2705
+ },
2706
+ {
2707
+ "epoch": 1.3050460654213507,
2708
+ "grad_norm": 0.502741813659668,
2709
+ "learning_rate": 2.8439847433918396e-06,
2710
+ "loss": 0.327,
2711
+ "step": 3860
2712
+ },
2713
+ {
2714
+ "epoch": 1.3084270137773646,
2715
+ "grad_norm": 0.46515053510665894,
2716
+ "learning_rate": 2.819309067710993e-06,
2717
+ "loss": 0.2677,
2718
+ "step": 3870
2719
+ },
2720
+ {
2721
+ "epoch": 1.3118079621333785,
2722
+ "grad_norm": 0.6406238675117493,
2723
+ "learning_rate": 2.794698806600331e-06,
2724
+ "loss": 0.2931,
2725
+ "step": 3880
2726
+ },
2727
+ {
2728
+ "epoch": 1.3151889104893923,
2729
+ "grad_norm": 0.43347349762916565,
2730
+ "learning_rate": 2.7701546982981903e-06,
2731
+ "loss": 0.234,
2732
+ "step": 3890
2733
+ },
2734
+ {
2735
+ "epoch": 1.318569858845406,
2736
+ "grad_norm": 0.4071926176548004,
2737
+ "learning_rate": 2.745677479058514e-06,
2738
+ "loss": 0.2896,
2739
+ "step": 3900
2740
+ },
2741
+ {
2742
+ "epoch": 1.32195080720142,
2743
+ "grad_norm": 0.515859067440033,
2744
+ "learning_rate": 2.7212678831287627e-06,
2745
+ "loss": 0.2456,
2746
+ "step": 3910
2747
+ },
2748
+ {
2749
+ "epoch": 1.3253317555574338,
2750
+ "grad_norm": 0.299369215965271,
2751
+ "learning_rate": 2.696926642727883e-06,
2752
+ "loss": 0.2698,
2753
+ "step": 3920
2754
+ },
2755
+ {
2756
+ "epoch": 1.3287127039134476,
2757
+ "grad_norm": 0.40438127517700195,
2758
+ "learning_rate": 2.6726544880243576e-06,
2759
+ "loss": 0.2391,
2760
+ "step": 3930
2761
+ },
2762
+ {
2763
+ "epoch": 1.3320936522694615,
2764
+ "grad_norm": 0.5045930743217468,
2765
+ "learning_rate": 2.6484521471142845e-06,
2766
+ "loss": 0.4502,
2767
+ "step": 3940
2768
+ },
2769
+ {
2770
+ "epoch": 1.3354746006254754,
2771
+ "grad_norm": 0.4972847104072571,
2772
+ "learning_rate": 2.6243203459995493e-06,
2773
+ "loss": 0.2635,
2774
+ "step": 3950
2775
+ },
2776
+ {
2777
+ "epoch": 1.3388555489814893,
2778
+ "grad_norm": 0.4043751657009125,
2779
+ "learning_rate": 2.600259808566045e-06,
2780
+ "loss": 0.2502,
2781
+ "step": 3960
2782
+ },
2783
+ {
2784
+ "epoch": 1.3422364973375032,
2785
+ "grad_norm": 0.38840949535369873,
2786
+ "learning_rate": 2.576271256561953e-06,
2787
+ "loss": 0.2075,
2788
+ "step": 3970
2789
+ },
2790
+ {
2791
+ "epoch": 1.345617445693517,
2792
+ "grad_norm": 0.45319676399230957,
2793
+ "learning_rate": 2.552355409576093e-06,
2794
+ "loss": 0.238,
2795
+ "step": 3980
2796
+ },
2797
+ {
2798
+ "epoch": 1.348998394049531,
2799
+ "grad_norm": 0.35230883955955505,
2800
+ "learning_rate": 2.5285129850163414e-06,
2801
+ "loss": 0.245,
2802
+ "step": 3990
2803
+ },
2804
+ {
2805
+ "epoch": 1.3523793424055448,
2806
+ "grad_norm": 0.4559631645679474,
2807
+ "learning_rate": 2.5047446980881106e-06,
2808
+ "loss": 0.2543,
2809
+ "step": 4000
2810
+ },
2811
+ {
2812
+ "epoch": 1.3557602907615587,
2813
+ "grad_norm": 0.4192922115325928,
2814
+ "learning_rate": 2.481051261772891e-06,
2815
+ "loss": 0.2605,
2816
+ "step": 4010
2817
+ },
2818
+ {
2819
+ "epoch": 1.3591412391175726,
2820
+ "grad_norm": 0.5145086050033569,
2821
+ "learning_rate": 2.45743338680687e-06,
2822
+ "loss": 0.2006,
2823
+ "step": 4020
2824
+ },
2825
+ {
2826
+ "epoch": 1.3625221874735862,
2827
+ "grad_norm": 0.6007429957389832,
2828
+ "learning_rate": 2.433891781659603e-06,
2829
+ "loss": 0.3686,
2830
+ "step": 4030
2831
+ },
2832
+ {
2833
+ "epoch": 1.3659031358296003,
2834
+ "grad_norm": 0.31601712107658386,
2835
+ "learning_rate": 2.4104271525127663e-06,
2836
+ "loss": 0.2496,
2837
+ "step": 4040
2838
+ },
2839
+ {
2840
+ "epoch": 1.369284084185614,
2841
+ "grad_norm": 0.5869673490524292,
2842
+ "learning_rate": 2.387040203238977e-06,
2843
+ "loss": 0.2735,
2844
+ "step": 4050
2845
+ },
2846
+ {
2847
+ "epoch": 1.3726650325416279,
2848
+ "grad_norm": 0.5303759574890137,
2849
+ "learning_rate": 2.363731635380673e-06,
2850
+ "loss": 0.3225,
2851
+ "step": 4060
2852
+ },
2853
+ {
2854
+ "epoch": 1.3760459808976417,
2855
+ "grad_norm": 0.528948962688446,
2856
+ "learning_rate": 2.3405021481290756e-06,
2857
+ "loss": 0.3101,
2858
+ "step": 4070
2859
+ },
2860
+ {
2861
+ "epoch": 1.3794269292536556,
2862
+ "grad_norm": 0.6139217615127563,
2863
+ "learning_rate": 2.317352438303202e-06,
2864
+ "loss": 0.2726,
2865
+ "step": 4080
2866
+ },
2867
+ {
2868
+ "epoch": 1.3828078776096695,
2869
+ "grad_norm": 0.32598358392715454,
2870
+ "learning_rate": 2.2942832003289823e-06,
2871
+ "loss": 0.2863,
2872
+ "step": 4090
2873
+ },
2874
+ {
2875
+ "epoch": 1.3861888259656834,
2876
+ "grad_norm": 0.5164593458175659,
2877
+ "learning_rate": 2.271295126218409e-06,
2878
+ "loss": 0.2956,
2879
+ "step": 4100
2880
+ },
2881
+ {
2882
+ "epoch": 1.3895697743216973,
2883
+ "grad_norm": 0.4716956317424774,
2884
+ "learning_rate": 2.248388905548794e-06,
2885
+ "loss": 0.2461,
2886
+ "step": 4110
2887
+ },
2888
+ {
2889
+ "epoch": 1.3929507226777111,
2890
+ "grad_norm": 0.40646469593048096,
2891
+ "learning_rate": 2.2255652254420774e-06,
2892
+ "loss": 0.2132,
2893
+ "step": 4120
2894
+ },
2895
+ {
2896
+ "epoch": 1.396331671033725,
2897
+ "grad_norm": 0.4522145688533783,
2898
+ "learning_rate": 2.2028247705442095e-06,
2899
+ "loss": 0.2326,
2900
+ "step": 4130
2901
+ },
2902
+ {
2903
+ "epoch": 1.399712619389739,
2904
+ "grad_norm": 0.4536775052547455,
2905
+ "learning_rate": 2.1801682230046256e-06,
2906
+ "loss": 0.2247,
2907
+ "step": 4140
2908
+ },
2909
+ {
2910
+ "epoch": 1.4030935677457528,
2911
+ "grad_norm": 0.44637754559516907,
2912
+ "learning_rate": 2.1575962624557754e-06,
2913
+ "loss": 0.333,
2914
+ "step": 4150
2915
+ },
2916
+ {
2917
+ "epoch": 1.4064745161017664,
2918
+ "grad_norm": 0.47522616386413574,
2919
+ "learning_rate": 2.1351095659927357e-06,
2920
+ "loss": 0.2266,
2921
+ "step": 4160
2922
+ },
2923
+ {
2924
+ "epoch": 1.4098554644577805,
2925
+ "grad_norm": 0.5105768442153931,
2926
+ "learning_rate": 2.1127088081529046e-06,
2927
+ "loss": 0.3008,
2928
+ "step": 4170
2929
+ },
2930
+ {
2931
+ "epoch": 1.4132364128137942,
2932
+ "grad_norm": 0.49943724274635315,
2933
+ "learning_rate": 2.09039466089576e-06,
2934
+ "loss": 0.2325,
2935
+ "step": 4180
2936
+ },
2937
+ {
2938
+ "epoch": 1.416617361169808,
2939
+ "grad_norm": 0.39756542444229126,
2940
+ "learning_rate": 2.0681677935827103e-06,
2941
+ "loss": 0.2085,
2942
+ "step": 4190
2943
+ },
2944
+ {
2945
+ "epoch": 1.419998309525822,
2946
+ "grad_norm": 0.6736156344413757,
2947
+ "learning_rate": 2.0460288729570122e-06,
2948
+ "loss": 0.2105,
2949
+ "step": 4200
2950
+ },
2951
+ {
2952
+ "epoch": 1.4233792578818358,
2953
+ "grad_norm": 0.4188987910747528,
2954
+ "learning_rate": 2.023978563123771e-06,
2955
+ "loss": 0.2956,
2956
+ "step": 4210
2957
+ },
2958
+ {
2959
+ "epoch": 1.4267602062378497,
2960
+ "grad_norm": 0.4170057475566864,
2961
+ "learning_rate": 2.0020175255300135e-06,
2962
+ "loss": 0.2368,
2963
+ "step": 4220
2964
+ },
2965
+ {
2966
+ "epoch": 1.4301411545938636,
2967
+ "grad_norm": 0.5500329732894897,
2968
+ "learning_rate": 1.9801464189448533e-06,
2969
+ "loss": 0.2739,
2970
+ "step": 4230
2971
+ },
2972
+ {
2973
+ "epoch": 1.4335221029498775,
2974
+ "grad_norm": 0.43552741408348083,
2975
+ "learning_rate": 1.958365899439731e-06,
2976
+ "loss": 0.3659,
2977
+ "step": 4240
2978
+ },
2979
+ {
2980
+ "epoch": 1.4369030513058914,
2981
+ "grad_norm": 0.459032267332077,
2982
+ "learning_rate": 1.9366766203687278e-06,
2983
+ "loss": 0.3526,
2984
+ "step": 4250
2985
+ },
2986
+ {
2987
+ "epoch": 1.4402839996619052,
2988
+ "grad_norm": 0.35360968112945557,
2989
+ "learning_rate": 1.915079232348972e-06,
2990
+ "loss": 0.3136,
2991
+ "step": 4260
2992
+ },
2993
+ {
2994
+ "epoch": 1.443664948017919,
2995
+ "grad_norm": 0.626098096370697,
2996
+ "learning_rate": 1.8935743832411163e-06,
2997
+ "loss": 0.3155,
2998
+ "step": 4270
2999
+ },
3000
+ {
3001
+ "epoch": 1.447045896373933,
3002
+ "grad_norm": 0.5543171763420105,
3003
+ "learning_rate": 1.8721627181299079e-06,
3004
+ "loss": 0.2791,
3005
+ "step": 4280
3006
+ },
3007
+ {
3008
+ "epoch": 1.4504268447299467,
3009
+ "grad_norm": 0.4099395275115967,
3010
+ "learning_rate": 1.8508448793048389e-06,
3011
+ "loss": 0.2423,
3012
+ "step": 4290
3013
+ },
3014
+ {
3015
+ "epoch": 1.4538077930859605,
3016
+ "grad_norm": 0.41322022676467896,
3017
+ "learning_rate": 1.8296215062408785e-06,
3018
+ "loss": 0.2153,
3019
+ "step": 4300
3020
+ },
3021
+ {
3022
+ "epoch": 1.4571887414419744,
3023
+ "grad_norm": 0.5144132375717163,
3024
+ "learning_rate": 1.8084932355792895e-06,
3025
+ "loss": 0.2694,
3026
+ "step": 4310
3027
+ },
3028
+ {
3029
+ "epoch": 1.4605696897979883,
3030
+ "grad_norm": 0.41947469115257263,
3031
+ "learning_rate": 1.787460701108527e-06,
3032
+ "loss": 0.2281,
3033
+ "step": 4320
3034
+ },
3035
+ {
3036
+ "epoch": 1.4639506381540022,
3037
+ "grad_norm": 0.47427940368652344,
3038
+ "learning_rate": 1.7665245337452368e-06,
3039
+ "loss": 0.2419,
3040
+ "step": 4330
3041
+ },
3042
+ {
3043
+ "epoch": 1.467331586510016,
3044
+ "grad_norm": 0.4865661859512329,
3045
+ "learning_rate": 1.7456853615153186e-06,
3046
+ "loss": 0.2989,
3047
+ "step": 4340
3048
+ },
3049
+ {
3050
+ "epoch": 1.47071253486603,
3051
+ "grad_norm": 0.45868703722953796,
3052
+ "learning_rate": 1.724943809535094e-06,
3053
+ "loss": 0.2557,
3054
+ "step": 4350
3055
+ },
3056
+ {
3057
+ "epoch": 1.4740934832220438,
3058
+ "grad_norm": 0.43730807304382324,
3059
+ "learning_rate": 1.7043004999925561e-06,
3060
+ "loss": 0.35,
3061
+ "step": 4360
3062
+ },
3063
+ {
3064
+ "epoch": 1.4774744315780577,
3065
+ "grad_norm": 0.5127968788146973,
3066
+ "learning_rate": 1.6837560521286955e-06,
3067
+ "loss": 0.2968,
3068
+ "step": 4370
3069
+ },
3070
+ {
3071
+ "epoch": 1.4808553799340716,
3072
+ "grad_norm": 0.5706360340118408,
3073
+ "learning_rate": 1.6633110822189358e-06,
3074
+ "loss": 0.3126,
3075
+ "step": 4380
3076
+ },
3077
+ {
3078
+ "epoch": 1.4842363282900854,
3079
+ "grad_norm": 0.49161526560783386,
3080
+ "learning_rate": 1.6429662035546451e-06,
3081
+ "loss": 0.2368,
3082
+ "step": 4390
3083
+ },
3084
+ {
3085
+ "epoch": 1.487617276646099,
3086
+ "grad_norm": 0.4524145722389221,
3087
+ "learning_rate": 1.62272202642473e-06,
3088
+ "loss": 0.2418,
3089
+ "step": 4400
3090
+ },
3091
+ {
3092
+ "epoch": 1.4909982250021132,
3093
+ "grad_norm": 0.5152844190597534,
3094
+ "learning_rate": 1.6025791580973433e-06,
3095
+ "loss": 0.3256,
3096
+ "step": 4410
3097
+ },
3098
+ {
3099
+ "epoch": 1.4943791733581269,
3100
+ "grad_norm": 0.8543993234634399,
3101
+ "learning_rate": 1.5825382028016516e-06,
3102
+ "loss": 0.2367,
3103
+ "step": 4420
3104
+ },
3105
+ {
3106
+ "epoch": 1.4977601217141407,
3107
+ "grad_norm": 0.48480215668678284,
3108
+ "learning_rate": 1.5625997617097244e-06,
3109
+ "loss": 0.3161,
3110
+ "step": 4430
3111
+ },
3112
+ {
3113
+ "epoch": 1.5011410700701546,
3114
+ "grad_norm": 0.32103753089904785,
3115
+ "learning_rate": 1.5427644329184931e-06,
3116
+ "loss": 0.3005,
3117
+ "step": 4440
3118
+ },
3119
+ {
3120
+ "epoch": 1.5045220184261685,
3121
+ "grad_norm": 0.36977288126945496,
3122
+ "learning_rate": 1.5230328114318127e-06,
3123
+ "loss": 0.2552,
3124
+ "step": 4450
3125
+ },
3126
+ {
3127
+ "epoch": 1.5079029667821824,
3128
+ "grad_norm": 0.6022387742996216,
3129
+ "learning_rate": 1.5034054891426081e-06,
3130
+ "loss": 0.4187,
3131
+ "step": 4460
3132
+ },
3133
+ {
3134
+ "epoch": 1.5112839151381963,
3135
+ "grad_norm": 0.541149377822876,
3136
+ "learning_rate": 1.4838830548151233e-06,
3137
+ "loss": 0.2491,
3138
+ "step": 4470
3139
+ },
3140
+ {
3141
+ "epoch": 1.5146648634942101,
3142
+ "grad_norm": 0.350790798664093,
3143
+ "learning_rate": 1.4644660940672628e-06,
3144
+ "loss": 0.2205,
3145
+ "step": 4480
3146
+ },
3147
+ {
3148
+ "epoch": 1.518045811850224,
3149
+ "grad_norm": 0.3898858428001404,
3150
+ "learning_rate": 1.4451551893530197e-06,
3151
+ "loss": 0.278,
3152
+ "step": 4490
3153
+ },
3154
+ {
3155
+ "epoch": 1.521426760206238,
3156
+ "grad_norm": 0.5894114375114441,
3157
+ "learning_rate": 1.4259509199450056e-06,
3158
+ "loss": 0.2869,
3159
+ "step": 4500
3160
+ },
3161
+ {
3162
+ "epoch": 1.5248077085622516,
3163
+ "grad_norm": 0.31155502796173096,
3164
+ "learning_rate": 1.4068538619170763e-06,
3165
+ "loss": 0.2791,
3166
+ "step": 4510
3167
+ },
3168
+ {
3169
+ "epoch": 1.5281886569182657,
3170
+ "grad_norm": 0.4483155906200409,
3171
+ "learning_rate": 1.387864588127044e-06,
3172
+ "loss": 0.3608,
3173
+ "step": 4520
3174
+ },
3175
+ {
3176
+ "epoch": 1.5315696052742793,
3177
+ "grad_norm": 0.5658784508705139,
3178
+ "learning_rate": 1.368983668199501e-06,
3179
+ "loss": 0.3057,
3180
+ "step": 4530
3181
+ },
3182
+ {
3183
+ "epoch": 1.5349505536302934,
3184
+ "grad_norm": 0.4309914708137512,
3185
+ "learning_rate": 1.3502116685087303e-06,
3186
+ "loss": 0.2691,
3187
+ "step": 4540
3188
+ },
3189
+ {
3190
+ "epoch": 1.538331501986307,
3191
+ "grad_norm": 0.4155011773109436,
3192
+ "learning_rate": 1.3315491521617146e-06,
3193
+ "loss": 0.3301,
3194
+ "step": 4550
3195
+ },
3196
+ {
3197
+ "epoch": 1.5417124503423212,
3198
+ "grad_norm": 0.5074723362922668,
3199
+ "learning_rate": 1.312996678981247e-06,
3200
+ "loss": 0.3004,
3201
+ "step": 4560
3202
+ },
3203
+ {
3204
+ "epoch": 1.5450933986983348,
3205
+ "grad_norm": 0.43595585227012634,
3206
+ "learning_rate": 1.2945548054891322e-06,
3207
+ "loss": 0.2954,
3208
+ "step": 4570
3209
+ },
3210
+ {
3211
+ "epoch": 1.5484743470543487,
3212
+ "grad_norm": 0.4728466868400574,
3213
+ "learning_rate": 1.276224084889502e-06,
3214
+ "loss": 0.2239,
3215
+ "step": 4580
3216
+ },
3217
+ {
3218
+ "epoch": 1.5518552954103626,
3219
+ "grad_norm": 0.639360249042511,
3220
+ "learning_rate": 1.2580050670522098e-06,
3221
+ "loss": 0.256,
3222
+ "step": 4590
3223
+ },
3224
+ {
3225
+ "epoch": 1.5552362437663765,
3226
+ "grad_norm": 0.5375871658325195,
3227
+ "learning_rate": 1.2398982984963454e-06,
3228
+ "loss": 0.2932,
3229
+ "step": 4600
3230
+ },
3231
+ {
3232
+ "epoch": 1.5586171921223904,
3233
+ "grad_norm": 0.5295315384864807,
3234
+ "learning_rate": 1.2219043223738375e-06,
3235
+ "loss": 0.2399,
3236
+ "step": 4610
3237
+ },
3238
+ {
3239
+ "epoch": 1.5619981404784042,
3240
+ "grad_norm": 0.4723894000053406,
3241
+ "learning_rate": 1.2040236784531562e-06,
3242
+ "loss": 0.213,
3243
+ "step": 4620
3244
+ },
3245
+ {
3246
+ "epoch": 1.5653790888344181,
3247
+ "grad_norm": 0.6368930339813232,
3248
+ "learning_rate": 1.186256903103129e-06,
3249
+ "loss": 0.3699,
3250
+ "step": 4630
3251
+ },
3252
+ {
3253
+ "epoch": 1.5687600371904318,
3254
+ "grad_norm": 0.5067258477210999,
3255
+ "learning_rate": 1.1686045292768488e-06,
3256
+ "loss": 0.2328,
3257
+ "step": 4640
3258
+ },
3259
+ {
3260
+ "epoch": 1.5721409855464459,
3261
+ "grad_norm": 0.4164457321166992,
3262
+ "learning_rate": 1.1510670864956813e-06,
3263
+ "loss": 0.2112,
3264
+ "step": 4650
3265
+ },
3266
+ {
3267
+ "epoch": 1.5755219339024595,
3268
+ "grad_norm": 0.4420374929904938,
3269
+ "learning_rate": 1.133645100833391e-06,
3270
+ "loss": 0.2912,
3271
+ "step": 4660
3272
+ },
3273
+ {
3274
+ "epoch": 1.5789028822584736,
3275
+ "grad_norm": 0.37962713837623596,
3276
+ "learning_rate": 1.116339094900349e-06,
3277
+ "loss": 0.3424,
3278
+ "step": 4670
3279
+ },
3280
+ {
3281
+ "epoch": 1.5822838306144873,
3282
+ "grad_norm": 0.42132097482681274,
3283
+ "learning_rate": 1.0991495878278663e-06,
3284
+ "loss": 0.2643,
3285
+ "step": 4680
3286
+ },
3287
+ {
3288
+ "epoch": 1.5856647789705012,
3289
+ "grad_norm": 0.4428631365299225,
3290
+ "learning_rate": 1.0820770952526155e-06,
3291
+ "loss": 0.3138,
3292
+ "step": 4690
3293
+ },
3294
+ {
3295
+ "epoch": 1.589045727326515,
3296
+ "grad_norm": 0.44889330863952637,
3297
+ "learning_rate": 1.0651221293011665e-06,
3298
+ "loss": 0.2902,
3299
+ "step": 4700
3300
+ },
3301
+ {
3302
+ "epoch": 1.592426675682529,
3303
+ "grad_norm": 0.4768747091293335,
3304
+ "learning_rate": 1.0482851985746196e-06,
3305
+ "loss": 0.3105,
3306
+ "step": 4710
3307
+ },
3308
+ {
3309
+ "epoch": 1.5958076240385428,
3310
+ "grad_norm": 0.4923536479473114,
3311
+ "learning_rate": 1.0315668081333519e-06,
3312
+ "loss": 0.2652,
3313
+ "step": 4720
3314
+ },
3315
+ {
3316
+ "epoch": 1.5991885723945567,
3317
+ "grad_norm": 0.4794210195541382,
3318
+ "learning_rate": 1.0149674594818676e-06,
3319
+ "loss": 0.2451,
3320
+ "step": 4730
3321
+ },
3322
+ {
3323
+ "epoch": 1.6025695207505706,
3324
+ "grad_norm": 0.4167225658893585,
3325
+ "learning_rate": 9.98487650553755e-07,
3326
+ "loss": 0.287,
3327
+ "step": 4740
3328
+ },
3329
+ {
3330
+ "epoch": 1.6059504691065842,
3331
+ "grad_norm": 0.45434409379959106,
3332
+ "learning_rate": 9.821278756967467e-07,
3333
+ "loss": 0.2529,
3334
+ "step": 4750
3335
+ },
3336
+ {
3337
+ "epoch": 1.6093314174625983,
3338
+ "grad_norm": 0.5816154479980469,
3339
+ "learning_rate": 9.658886256578914e-07,
3340
+ "loss": 0.2542,
3341
+ "step": 4760
3342
+ },
3343
+ {
3344
+ "epoch": 1.612712365818612,
3345
+ "grad_norm": 0.5081009864807129,
3346
+ "learning_rate": 9.49770387568833e-07,
3347
+ "loss": 0.2435,
3348
+ "step": 4770
3349
+ },
3350
+ {
3351
+ "epoch": 1.616093314174626,
3352
+ "grad_norm": 0.4589058458805084,
3353
+ "learning_rate": 9.337736449312007e-07,
3354
+ "loss": 0.2331,
3355
+ "step": 4780
3356
+ },
3357
+ {
3358
+ "epoch": 1.6194742625306398,
3359
+ "grad_norm": 0.3296624422073364,
3360
+ "learning_rate": 9.178988776021024e-07,
3361
+ "loss": 0.2994,
3362
+ "step": 4790
3363
+ },
3364
+ {
3365
+ "epoch": 1.6228552108866539,
3366
+ "grad_norm": 0.39514023065567017,
3367
+ "learning_rate": 9.021465617797331e-07,
3368
+ "loss": 0.2635,
3369
+ "step": 4800
3370
+ },
3371
+ {
3372
+ "epoch": 1.6262361592426675,
3373
+ "grad_norm": 0.3601683974266052,
3374
+ "learning_rate": 8.865171699890835e-07,
3375
+ "loss": 0.2796,
3376
+ "step": 4810
3377
+ },
3378
+ {
3379
+ "epoch": 1.6296171075986814,
3380
+ "grad_norm": 0.38344496488571167,
3381
+ "learning_rate": 8.710111710677754e-07,
3382
+ "loss": 0.2747,
3383
+ "step": 4820
3384
+ },
3385
+ {
3386
+ "epoch": 1.6329980559546953,
3387
+ "grad_norm": 0.671013355255127,
3388
+ "learning_rate": 8.556290301519892e-07,
3389
+ "loss": 0.2393,
3390
+ "step": 4830
3391
+ },
3392
+ {
3393
+ "epoch": 1.6363790043107092,
3394
+ "grad_norm": 0.4742591977119446,
3395
+ "learning_rate": 8.403712086625176e-07,
3396
+ "loss": 0.2474,
3397
+ "step": 4840
3398
+ },
3399
+ {
3400
+ "epoch": 1.639759952666723,
3401
+ "grad_norm": 0.5117056369781494,
3402
+ "learning_rate": 8.252381642909218e-07,
3403
+ "loss": 0.2232,
3404
+ "step": 4850
3405
+ },
3406
+ {
3407
+ "epoch": 1.643140901022737,
3408
+ "grad_norm": 0.4941207766532898,
3409
+ "learning_rate": 8.102303509857984e-07,
3410
+ "loss": 0.2465,
3411
+ "step": 4860
3412
+ },
3413
+ {
3414
+ "epoch": 1.6465218493787508,
3415
+ "grad_norm": 0.5719410181045532,
3416
+ "learning_rate": 7.953482189391687e-07,
3417
+ "loss": 0.2633,
3418
+ "step": 4870
3419
+ },
3420
+ {
3421
+ "epoch": 1.6499027977347644,
3422
+ "grad_norm": 0.4885143041610718,
3423
+ "learning_rate": 7.805922145729716e-07,
3424
+ "loss": 0.2514,
3425
+ "step": 4880
3426
+ },
3427
+ {
3428
+ "epoch": 1.6532837460907785,
3429
+ "grad_norm": 0.38283446431159973,
3430
+ "learning_rate": 7.65962780525668e-07,
3431
+ "loss": 0.299,
3432
+ "step": 4890
3433
+ },
3434
+ {
3435
+ "epoch": 1.6566646944467922,
3436
+ "grad_norm": 0.4202812910079956,
3437
+ "learning_rate": 7.514603556389716e-07,
3438
+ "loss": 0.2738,
3439
+ "step": 4900
3440
+ },
3441
+ {
3442
+ "epoch": 1.6600456428028063,
3443
+ "grad_norm": 0.4934067130088806,
3444
+ "learning_rate": 7.370853749446743e-07,
3445
+ "loss": 0.2809,
3446
+ "step": 4910
3447
+ },
3448
+ {
3449
+ "epoch": 1.66342659115882,
3450
+ "grad_norm": 0.4023735821247101,
3451
+ "learning_rate": 7.228382696516078e-07,
3452
+ "loss": 0.2402,
3453
+ "step": 4920
3454
+ },
3455
+ {
3456
+ "epoch": 1.666807539514834,
3457
+ "grad_norm": 0.41086632013320923,
3458
+ "learning_rate": 7.087194671326986e-07,
3459
+ "loss": 0.2025,
3460
+ "step": 4930
3461
+ },
3462
+ {
3463
+ "epoch": 1.6701884878708477,
3464
+ "grad_norm": 0.4808848798274994,
3465
+ "learning_rate": 6.947293909121561e-07,
3466
+ "loss": 0.2326,
3467
+ "step": 4940
3468
+ },
3469
+ {
3470
+ "epoch": 1.6735694362268616,
3471
+ "grad_norm": 0.5335634350776672,
3472
+ "learning_rate": 6.80868460652761e-07,
3473
+ "loss": 0.2708,
3474
+ "step": 4950
3475
+ },
3476
+ {
3477
+ "epoch": 1.6769503845828755,
3478
+ "grad_norm": 0.411200612783432,
3479
+ "learning_rate": 6.67137092143279e-07,
3480
+ "loss": 0.2139,
3481
+ "step": 4960
3482
+ },
3483
+ {
3484
+ "epoch": 1.6803313329388894,
3485
+ "grad_norm": 0.44955313205718994,
3486
+ "learning_rate": 6.53535697285993e-07,
3487
+ "loss": 0.258,
3488
+ "step": 4970
3489
+ },
3490
+ {
3491
+ "epoch": 1.6837122812949032,
3492
+ "grad_norm": 0.4411759078502655,
3493
+ "learning_rate": 6.400646840843406e-07,
3494
+ "loss": 0.308,
3495
+ "step": 4980
3496
+ },
3497
+ {
3498
+ "epoch": 1.6870932296509171,
3499
+ "grad_norm": 0.5405946969985962,
3500
+ "learning_rate": 6.267244566306801e-07,
3501
+ "loss": 0.2483,
3502
+ "step": 4990
3503
+ },
3504
+ {
3505
+ "epoch": 1.690474178006931,
3506
+ "grad_norm": 0.661658525466919,
3507
+ "learning_rate": 6.135154150941647e-07,
3508
+ "loss": 0.2629,
3509
+ "step": 5000
3510
+ }
3511
+ ],
3512
+ "logging_steps": 10,
3513
+ "max_steps": 5914,
3514
+ "num_input_tokens_seen": 0,
3515
+ "num_train_epochs": 2,
3516
+ "save_steps": 1000,
3517
+ "stateful_callbacks": {
3518
+ "TrainerControl": {
3519
+ "args": {
3520
+ "should_epoch_stop": false,
3521
+ "should_evaluate": false,
3522
+ "should_log": false,
3523
+ "should_save": true,
3524
+ "should_training_stop": false
3525
+ },
3526
+ "attributes": {}
3527
+ }
3528
+ },
3529
+ "total_flos": 7.856156629748679e+18,
3530
+ "train_batch_size": 4,
3531
+ "trial_name": null,
3532
+ "trial_params": null
3533
+ }
checkpoint-5000/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:107cb538919cca7e971dd8435372df02aebcbc36dec79e97d1add154e8c71a99
3
+ size 6289
checkpoint-5914/README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: microsoft/Phi-3.5-vision-instruct
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:microsoft/Phi-3.5-vision-instruct
7
+ - lora
8
+ - transformers
9
+ ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+
16
+
17
+ ## Model Details
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+
24
+
25
+ - **Developed by:** [More Information Needed]
26
+ - **Funded by [optional]:** [More Information Needed]
27
+ - **Shared by [optional]:** [More Information Needed]
28
+ - **Model type:** [More Information Needed]
29
+ - **Language(s) (NLP):** [More Information Needed]
30
+ - **License:** [More Information Needed]
31
+ - **Finetuned from model [optional]:** [More Information Needed]
32
+
33
+ ### Model Sources [optional]
34
+
35
+ <!-- Provide the basic links for the model. -->
36
+
37
+ - **Repository:** [More Information Needed]
38
+ - **Paper [optional]:** [More Information Needed]
39
+ - **Demo [optional]:** [More Information Needed]
40
+
41
+ ## Uses
42
+
43
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
44
+
45
+ ### Direct Use
46
+
47
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
48
+
49
+ [More Information Needed]
50
+
51
+ ### Downstream Use [optional]
52
+
53
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
54
+
55
+ [More Information Needed]
56
+
57
+ ### Out-of-Scope Use
58
+
59
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
60
+
61
+ [More Information Needed]
62
+
63
+ ## Bias, Risks, and Limitations
64
+
65
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
66
+
67
+ [More Information Needed]
68
+
69
+ ### Recommendations
70
+
71
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
72
+
73
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
74
+
75
+ ## How to Get Started with the Model
76
+
77
+ Use the code below to get started with the model.
78
+
79
+ [More Information Needed]
80
+
81
+ ## Training Details
82
+
83
+ ### Training Data
84
+
85
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
86
+
87
+ [More Information Needed]
88
+
89
+ ### Training Procedure
90
+
91
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
92
+
93
+ #### Preprocessing [optional]
94
+
95
+ [More Information Needed]
96
+
97
+
98
+ #### Training Hyperparameters
99
+
100
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
101
+
102
+ #### Speeds, Sizes, Times [optional]
103
+
104
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
105
+
106
+ [More Information Needed]
107
+
108
+ ## Evaluation
109
+
110
+ <!-- This section describes the evaluation protocols and provides the results. -->
111
+
112
+ ### Testing Data, Factors & Metrics
113
+
114
+ #### Testing Data
115
+
116
+ <!-- This should link to a Dataset Card if possible. -->
117
+
118
+ [More Information Needed]
119
+
120
+ #### Factors
121
+
122
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
123
+
124
+ [More Information Needed]
125
+
126
+ #### Metrics
127
+
128
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
129
+
130
+ [More Information Needed]
131
+
132
+ ### Results
133
+
134
+ [More Information Needed]
135
+
136
+ #### Summary
137
+
138
+
139
+
140
+ ## Model Examination [optional]
141
+
142
+ <!-- Relevant interpretability work for the model goes here -->
143
+
144
+ [More Information Needed]
145
+
146
+ ## Environmental Impact
147
+
148
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
149
+
150
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
151
+
152
+ - **Hardware Type:** [More Information Needed]
153
+ - **Hours used:** [More Information Needed]
154
+ - **Cloud Provider:** [More Information Needed]
155
+ - **Compute Region:** [More Information Needed]
156
+ - **Carbon Emitted:** [More Information Needed]
157
+
158
+ ## Technical Specifications [optional]
159
+
160
+ ### Model Architecture and Objective
161
+
162
+ [More Information Needed]
163
+
164
+ ### Compute Infrastructure
165
+
166
+ [More Information Needed]
167
+
168
+ #### Hardware
169
+
170
+ [More Information Needed]
171
+
172
+ #### Software
173
+
174
+ [More Information Needed]
175
+
176
+ ## Citation [optional]
177
+
178
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
179
+
180
+ **BibTeX:**
181
+
182
+ [More Information Needed]
183
+
184
+ **APA:**
185
+
186
+ [More Information Needed]
187
+
188
+ ## Glossary [optional]
189
+
190
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
191
+
192
+ [More Information Needed]
193
+
194
+ ## More Information [optional]
195
+
196
+ [More Information Needed]
197
+
198
+ ## Model Card Authors [optional]
199
+
200
+ [More Information Needed]
201
+
202
+ ## Model Card Contact
203
+
204
+ [More Information Needed]
205
+ ### Framework versions
206
+
207
+ - PEFT 0.18.1
checkpoint-5914/adapter_config.json ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "microsoft/Phi-3.5-vision-instruct",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": false,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "layer_replication": null,
16
+ "layers_pattern": null,
17
+ "layers_to_transform": null,
18
+ "loftq_config": {},
19
+ "lora_alpha": 128,
20
+ "lora_bias": false,
21
+ "lora_dropout": 0.05,
22
+ "megatron_config": null,
23
+ "megatron_core": "megatron.core",
24
+ "modules_to_save": null,
25
+ "peft_type": "LORA",
26
+ "peft_version": "0.18.1",
27
+ "qalora_group_size": 16,
28
+ "r": 64,
29
+ "rank_pattern": {},
30
+ "revision": null,
31
+ "target_modules": [
32
+ "down_proj",
33
+ "qkv_proj",
34
+ "gate_up_proj",
35
+ "o_proj"
36
+ ],
37
+ "target_parameters": null,
38
+ "task_type": "CAUSAL_LM",
39
+ "trainable_token_indices": null,
40
+ "use_dora": false,
41
+ "use_qalora": false,
42
+ "use_rslora": false
43
+ }
checkpoint-5914/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d67c56376e70283e205d2850989edf7759aa866cebc1c9d59eba61dcc9830d97
3
+ size 402688040
checkpoint-5914/chat_template.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "chat_template": "{% for message in messages %}{{'<|' + message['role'] + '|>' + '\n' + message['content'] + '<|end|>\n' }}{% endfor %}{% if add_generation_prompt and messages[-1]['role'] != 'assistant' %}{{- '<|assistant|>\n' -}}{% endif %}"
3
+ }
checkpoint-5914/non_lora_state_dict.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2043c04dab255ee33d8177cc5145d0395d45cb9af4bc0bf51ea58b5c7b901448
3
+ size 8293470415
checkpoint-5914/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:30c75a2b981e799335881a8a55b0656c105fce807ba40e020c537f8886d1bb76
3
+ size 805522635
checkpoint-5914/preprocessor_config.json ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "auto_map": {
3
+ "AutoImageProcessor": "microsoft/Phi-3.5-vision-instruct--processing_phi3_v.Phi3VImageProcessor",
4
+ "AutoProcessor": "microsoft/Phi-3.5-vision-instruct--processing_phi3_v.Phi3VProcessor"
5
+ },
6
+ "do_convert_rgb": true,
7
+ "image_mean": [
8
+ 0.48145466,
9
+ 0.4578275,
10
+ 0.40821073
11
+ ],
12
+ "image_processor_type": "Phi3VImageProcessor",
13
+ "image_std": [
14
+ 0.26862954,
15
+ 0.26130258,
16
+ 0.27577711
17
+ ],
18
+ "num_crops": 16,
19
+ "num_img_tokens": 144,
20
+ "processor_class": "Phi3VProcessor"
21
+ }
checkpoint-5914/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7b32535370af2ef9b4220fee9fcdc2254c8b9f197ed014fc32ed1c88d89bac43
3
+ size 14709
checkpoint-5914/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7ee41684848e0788fd24e2f9421fa0b1a52ec3c8e698c30eb2181b7db713194f
3
+ size 1465
checkpoint-5914/special_tokens_map.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "additional_special_tokens": [
3
+ "<|system|>",
4
+ "<|end|>",
5
+ "<|user|>",
6
+ "<|end|>"
7
+ ],
8
+ "bos_token": {
9
+ "content": "<s>",
10
+ "lstrip": false,
11
+ "normalized": false,
12
+ "rstrip": false,
13
+ "single_word": false
14
+ },
15
+ "eos_token": {
16
+ "content": "<|endoftext|>",
17
+ "lstrip": false,
18
+ "normalized": false,
19
+ "rstrip": false,
20
+ "single_word": false
21
+ },
22
+ "pad_token": "<unk>",
23
+ "unk_token": {
24
+ "content": "<unk>",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ }
30
+ }
checkpoint-5914/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-5914/tokenizer_config.json ADDED
@@ -0,0 +1,414 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_bos_token": true,
3
+ "add_eos_token": false,
4
+ "add_prefix_space": null,
5
+ "added_tokens_decoder": {
6
+ "0": {
7
+ "content": "<unk>",
8
+ "lstrip": false,
9
+ "normalized": false,
10
+ "rstrip": false,
11
+ "single_word": false,
12
+ "special": true
13
+ },
14
+ "1": {
15
+ "content": "<s>",
16
+ "lstrip": false,
17
+ "normalized": false,
18
+ "rstrip": false,
19
+ "single_word": false,
20
+ "special": true
21
+ },
22
+ "2": {
23
+ "content": "</s>",
24
+ "lstrip": false,
25
+ "normalized": false,
26
+ "rstrip": true,
27
+ "single_word": false,
28
+ "special": false
29
+ },
30
+ "32000": {
31
+ "content": "<|endoftext|>",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false,
36
+ "special": true
37
+ },
38
+ "32001": {
39
+ "content": "<|assistant|>",
40
+ "lstrip": false,
41
+ "normalized": false,
42
+ "rstrip": true,
43
+ "single_word": false,
44
+ "special": true
45
+ },
46
+ "32002": {
47
+ "content": "<|placeholder1|>",
48
+ "lstrip": false,
49
+ "normalized": false,
50
+ "rstrip": true,
51
+ "single_word": false,
52
+ "special": true
53
+ },
54
+ "32003": {
55
+ "content": "<|placeholder2|>",
56
+ "lstrip": false,
57
+ "normalized": false,
58
+ "rstrip": true,
59
+ "single_word": false,
60
+ "special": true
61
+ },
62
+ "32004": {
63
+ "content": "<|placeholder3|>",
64
+ "lstrip": false,
65
+ "normalized": false,
66
+ "rstrip": true,
67
+ "single_word": false,
68
+ "special": true
69
+ },
70
+ "32005": {
71
+ "content": "<|placeholder4|>",
72
+ "lstrip": false,
73
+ "normalized": false,
74
+ "rstrip": true,
75
+ "single_word": false,
76
+ "special": true
77
+ },
78
+ "32006": {
79
+ "content": "<|system|>",
80
+ "lstrip": false,
81
+ "normalized": false,
82
+ "rstrip": false,
83
+ "single_word": false,
84
+ "special": true
85
+ },
86
+ "32007": {
87
+ "content": "<|end|>",
88
+ "lstrip": false,
89
+ "normalized": false,
90
+ "rstrip": false,
91
+ "single_word": false,
92
+ "special": true
93
+ },
94
+ "32008": {
95
+ "content": "<|placeholder5|>",
96
+ "lstrip": false,
97
+ "normalized": false,
98
+ "rstrip": true,
99
+ "single_word": false,
100
+ "special": true
101
+ },
102
+ "32009": {
103
+ "content": "<|placeholder6|>",
104
+ "lstrip": false,
105
+ "normalized": false,
106
+ "rstrip": true,
107
+ "single_word": false,
108
+ "special": true
109
+ },
110
+ "32010": {
111
+ "content": "<|user|>",
112
+ "lstrip": false,
113
+ "normalized": false,
114
+ "rstrip": false,
115
+ "single_word": false,
116
+ "special": true
117
+ },
118
+ "32011": {
119
+ "content": "<|placeholder7|>",
120
+ "lstrip": false,
121
+ "normalized": false,
122
+ "rstrip": true,
123
+ "single_word": false,
124
+ "special": true
125
+ },
126
+ "32012": {
127
+ "content": "<|placeholder8|>",
128
+ "lstrip": false,
129
+ "normalized": false,
130
+ "rstrip": true,
131
+ "single_word": false,
132
+ "special": true
133
+ },
134
+ "32013": {
135
+ "content": "<|placeholder9|>",
136
+ "lstrip": false,
137
+ "normalized": false,
138
+ "rstrip": true,
139
+ "single_word": false,
140
+ "special": true
141
+ },
142
+ "32014": {
143
+ "content": "<|placeholder10|>",
144
+ "lstrip": false,
145
+ "normalized": false,
146
+ "rstrip": true,
147
+ "single_word": false,
148
+ "special": true
149
+ },
150
+ "32015": {
151
+ "content": "<|placeholder11|>",
152
+ "lstrip": false,
153
+ "normalized": false,
154
+ "rstrip": true,
155
+ "single_word": false,
156
+ "special": true
157
+ },
158
+ "32016": {
159
+ "content": "<|placeholder12|>",
160
+ "lstrip": false,
161
+ "normalized": false,
162
+ "rstrip": true,
163
+ "single_word": false,
164
+ "special": true
165
+ },
166
+ "32017": {
167
+ "content": "<|placeholder13|>",
168
+ "lstrip": false,
169
+ "normalized": false,
170
+ "rstrip": true,
171
+ "single_word": false,
172
+ "special": true
173
+ },
174
+ "32018": {
175
+ "content": "<|placeholder14|>",
176
+ "lstrip": false,
177
+ "normalized": false,
178
+ "rstrip": true,
179
+ "single_word": false,
180
+ "special": true
181
+ },
182
+ "32019": {
183
+ "content": "<|placeholder15|>",
184
+ "lstrip": false,
185
+ "normalized": false,
186
+ "rstrip": true,
187
+ "single_word": false,
188
+ "special": true
189
+ },
190
+ "32020": {
191
+ "content": "<|placeholder16|>",
192
+ "lstrip": false,
193
+ "normalized": false,
194
+ "rstrip": true,
195
+ "single_word": false,
196
+ "special": true
197
+ },
198
+ "32021": {
199
+ "content": "<|placeholder17|>",
200
+ "lstrip": false,
201
+ "normalized": false,
202
+ "rstrip": true,
203
+ "single_word": false,
204
+ "special": true
205
+ },
206
+ "32022": {
207
+ "content": "<|placeholder18|>",
208
+ "lstrip": false,
209
+ "normalized": false,
210
+ "rstrip": true,
211
+ "single_word": false,
212
+ "special": true
213
+ },
214
+ "32023": {
215
+ "content": "<|placeholder19|>",
216
+ "lstrip": false,
217
+ "normalized": false,
218
+ "rstrip": true,
219
+ "single_word": false,
220
+ "special": true
221
+ },
222
+ "32024": {
223
+ "content": "<|placeholder20|>",
224
+ "lstrip": false,
225
+ "normalized": false,
226
+ "rstrip": true,
227
+ "single_word": false,
228
+ "special": true
229
+ },
230
+ "32025": {
231
+ "content": "<|placeholder21|>",
232
+ "lstrip": false,
233
+ "normalized": false,
234
+ "rstrip": true,
235
+ "single_word": false,
236
+ "special": true
237
+ },
238
+ "32026": {
239
+ "content": "<|placeholder22|>",
240
+ "lstrip": false,
241
+ "normalized": false,
242
+ "rstrip": true,
243
+ "single_word": false,
244
+ "special": true
245
+ },
246
+ "32027": {
247
+ "content": "<|placeholder23|>",
248
+ "lstrip": false,
249
+ "normalized": false,
250
+ "rstrip": true,
251
+ "single_word": false,
252
+ "special": true
253
+ },
254
+ "32028": {
255
+ "content": "<|placeholder24|>",
256
+ "lstrip": false,
257
+ "normalized": false,
258
+ "rstrip": true,
259
+ "single_word": false,
260
+ "special": true
261
+ },
262
+ "32029": {
263
+ "content": "<|placeholder25|>",
264
+ "lstrip": false,
265
+ "normalized": false,
266
+ "rstrip": true,
267
+ "single_word": false,
268
+ "special": true
269
+ },
270
+ "32030": {
271
+ "content": "<|placeholder26|>",
272
+ "lstrip": false,
273
+ "normalized": false,
274
+ "rstrip": true,
275
+ "single_word": false,
276
+ "special": true
277
+ },
278
+ "32031": {
279
+ "content": "<|placeholder27|>",
280
+ "lstrip": false,
281
+ "normalized": false,
282
+ "rstrip": true,
283
+ "single_word": false,
284
+ "special": true
285
+ },
286
+ "32032": {
287
+ "content": "<|placeholder28|>",
288
+ "lstrip": false,
289
+ "normalized": false,
290
+ "rstrip": true,
291
+ "single_word": false,
292
+ "special": true
293
+ },
294
+ "32033": {
295
+ "content": "<|placeholder29|>",
296
+ "lstrip": false,
297
+ "normalized": false,
298
+ "rstrip": true,
299
+ "single_word": false,
300
+ "special": true
301
+ },
302
+ "32034": {
303
+ "content": "<|placeholder30|>",
304
+ "lstrip": false,
305
+ "normalized": false,
306
+ "rstrip": true,
307
+ "single_word": false,
308
+ "special": true
309
+ },
310
+ "32035": {
311
+ "content": "<|placeholder31|>",
312
+ "lstrip": false,
313
+ "normalized": false,
314
+ "rstrip": true,
315
+ "single_word": false,
316
+ "special": true
317
+ },
318
+ "32036": {
319
+ "content": "<|placeholder32|>",
320
+ "lstrip": false,
321
+ "normalized": false,
322
+ "rstrip": true,
323
+ "single_word": false,
324
+ "special": true
325
+ },
326
+ "32037": {
327
+ "content": "<|placeholder33|>",
328
+ "lstrip": false,
329
+ "normalized": false,
330
+ "rstrip": true,
331
+ "single_word": false,
332
+ "special": true
333
+ },
334
+ "32038": {
335
+ "content": "<|placeholder34|>",
336
+ "lstrip": false,
337
+ "normalized": false,
338
+ "rstrip": true,
339
+ "single_word": false,
340
+ "special": true
341
+ },
342
+ "32039": {
343
+ "content": "<|placeholder35|>",
344
+ "lstrip": false,
345
+ "normalized": false,
346
+ "rstrip": true,
347
+ "single_word": false,
348
+ "special": true
349
+ },
350
+ "32040": {
351
+ "content": "<|placeholder36|>",
352
+ "lstrip": false,
353
+ "normalized": false,
354
+ "rstrip": true,
355
+ "single_word": false,
356
+ "special": true
357
+ },
358
+ "32041": {
359
+ "content": "<|placeholder37|>",
360
+ "lstrip": false,
361
+ "normalized": false,
362
+ "rstrip": true,
363
+ "single_word": false,
364
+ "special": true
365
+ },
366
+ "32042": {
367
+ "content": "<|placeholder38|>",
368
+ "lstrip": false,
369
+ "normalized": false,
370
+ "rstrip": true,
371
+ "single_word": false,
372
+ "special": true
373
+ },
374
+ "32043": {
375
+ "content": "<|placeholder39|>",
376
+ "lstrip": false,
377
+ "normalized": false,
378
+ "rstrip": true,
379
+ "single_word": false,
380
+ "special": true
381
+ },
382
+ "32044": {
383
+ "content": "<|image|>",
384
+ "lstrip": false,
385
+ "normalized": false,
386
+ "rstrip": true,
387
+ "single_word": false,
388
+ "special": true
389
+ }
390
+ },
391
+ "additional_special_tokens": [
392
+ "<|system|>",
393
+ "<|end|>",
394
+ "<|user|>",
395
+ "<|end|>"
396
+ ],
397
+ "auto_map": {
398
+ "AutoProcessor": "microsoft/Phi-3.5-vision-instruct--processing_phi3_v.Phi3VProcessor"
399
+ },
400
+ "bos_token": "<s>",
401
+ "chat_template": "{% for message in messages %}{{'<|' + message['role'] + '|>' + '\n' + message['content'] + '<|end|>\n' }}{% endfor %}{% if add_generation_prompt and messages[-1]['role'] != 'assistant' %}{{- '<|assistant|>\n' -}}{% endif %}",
402
+ "clean_up_tokenization_spaces": false,
403
+ "eos_token": "<|endoftext|>",
404
+ "legacy": false,
405
+ "model_max_length": 2048,
406
+ "num_crops": 16,
407
+ "pad_token": "<unk>",
408
+ "padding_side": "right",
409
+ "processor_class": "Phi3VProcessor",
410
+ "sp_model_kwargs": {},
411
+ "tokenizer_class": "LlamaTokenizer",
412
+ "unk_token": "<unk>",
413
+ "use_default_system_prompt": false
414
+ }
checkpoint-5914/trainer_state.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-5914/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:107cb538919cca7e971dd8435372df02aebcbc36dec79e97d1add154e8c71a99
3
+ size 6289
config.json ADDED
@@ -0,0 +1,155 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_attn_implementation_autoset": true,
3
+ "_name_or_path": "microsoft/Phi-3.5-vision-instruct",
4
+ "architectures": [
5
+ "Phi3VForCausalLM"
6
+ ],
7
+ "attention_dropout": 0.0,
8
+ "auto_map": {
9
+ "AutoConfig": "microsoft/Phi-3.5-vision-instruct--configuration_phi3_v.Phi3VConfig",
10
+ "AutoModelForCausalLM": "microsoft/Phi-3.5-vision-instruct--modeling_phi3_v.Phi3VForCausalLM"
11
+ },
12
+ "bos_token_id": 1,
13
+ "embd_layer": {
14
+ "embedding_cls": "image",
15
+ "hd_transform_order": "sub_glb",
16
+ "projection_cls": "mlp",
17
+ "use_hd_transform": true,
18
+ "with_learnable_separator": true
19
+ },
20
+ "embd_pdrop": 0.0,
21
+ "eos_token_id": 2,
22
+ "hidden_act": "silu",
23
+ "hidden_size": 3072,
24
+ "img_processor": {
25
+ "image_dim_out": 1024,
26
+ "model_name": "openai/clip-vit-large-patch14-336",
27
+ "name": "clip_vision_model",
28
+ "num_img_tokens": 144
29
+ },
30
+ "initializer_range": 0.02,
31
+ "intermediate_size": 8192,
32
+ "max_position_embeddings": 131072,
33
+ "model_type": "phi3_v",
34
+ "num_attention_heads": 32,
35
+ "num_hidden_layers": 32,
36
+ "num_key_value_heads": 32,
37
+ "original_max_position_embeddings": 4096,
38
+ "pad_token_id": 32000,
39
+ "projector_lr": null,
40
+ "resid_pdrop": 0.0,
41
+ "rms_norm_eps": 1e-05,
42
+ "rope_scaling": {
43
+ "long_factor": [
44
+ 1.0800000429153442,
45
+ 1.1100000143051147,
46
+ 1.1399999856948853,
47
+ 1.340000033378601,
48
+ 1.5899999141693115,
49
+ 1.600000023841858,
50
+ 1.6200000047683716,
51
+ 2.620000123977661,
52
+ 3.2300000190734863,
53
+ 3.2300000190734863,
54
+ 4.789999961853027,
55
+ 7.400000095367432,
56
+ 7.700000286102295,
57
+ 9.09000015258789,
58
+ 12.199999809265137,
59
+ 17.670000076293945,
60
+ 24.46000099182129,
61
+ 28.57000160217285,
62
+ 30.420001983642578,
63
+ 30.840002059936523,
64
+ 32.590003967285156,
65
+ 32.93000411987305,
66
+ 42.320003509521484,
67
+ 44.96000289916992,
68
+ 50.340003967285156,
69
+ 50.45000457763672,
70
+ 57.55000305175781,
71
+ 57.93000411987305,
72
+ 58.21000289916992,
73
+ 60.1400032043457,
74
+ 62.61000442504883,
75
+ 62.62000274658203,
76
+ 62.71000289916992,
77
+ 63.1400032043457,
78
+ 63.1400032043457,
79
+ 63.77000427246094,
80
+ 63.93000411987305,
81
+ 63.96000289916992,
82
+ 63.970001220703125,
83
+ 64.02999877929688,
84
+ 64.06999969482422,
85
+ 64.08000183105469,
86
+ 64.12000274658203,
87
+ 64.41000366210938,
88
+ 64.4800033569336,
89
+ 64.51000213623047,
90
+ 64.52999877929688,
91
+ 64.83999633789062
92
+ ],
93
+ "short_factor": [
94
+ 1.08,
95
+ 1.1,
96
+ 1.1300000000000001,
97
+ 1.2800000000000002,
98
+ 1.3100000000000003,
99
+ 1.4500000000000004,
100
+ 1.4500000000000004,
101
+ 1.9500000000000008,
102
+ 2.030000000000001,
103
+ 2.4299999999999926,
104
+ 2.5699999999999896,
105
+ 2.9499999999999815,
106
+ 3.729999999999965,
107
+ 3.869999999999962,
108
+ 4.189999999999955,
109
+ 4.43999999999995,
110
+ 4.6399999999999455,
111
+ 4.979999999999938,
112
+ 5.159999999999934,
113
+ 5.279999999999932,
114
+ 5.759999999999922,
115
+ 5.889999999999919,
116
+ 5.889999999999919,
117
+ 5.969999999999917,
118
+ 6.089999999999915,
119
+ 6.2799999999999105,
120
+ 6.7699999999999,
121
+ 6.8899999999998975,
122
+ 7.109999999999893,
123
+ 7.129999999999892,
124
+ 7.179999999999891,
125
+ 7.289999999999889,
126
+ 7.339999999999888,
127
+ 7.559999999999883,
128
+ 7.619999999999882,
129
+ 7.69999999999988,
130
+ 7.879999999999876,
131
+ 7.879999999999876,
132
+ 7.879999999999876,
133
+ 7.939999999999875,
134
+ 7.949999999999875,
135
+ 7.979999999999874,
136
+ 8.19999999999987,
137
+ 8.439999999999864,
138
+ 8.469999999999864,
139
+ 8.589999999999861,
140
+ 8.809999999999857,
141
+ 8.999999999999853
142
+ ],
143
+ "type": "su"
144
+ },
145
+ "rope_theta": 10000.0,
146
+ "sliding_window": 262144,
147
+ "tie_word_embeddings": false,
148
+ "tokenizer_model_max_length": 2048,
149
+ "tokenizer_padding_side": "right",
150
+ "torch_dtype": "bfloat16",
151
+ "transformers_version": "4.46.0",
152
+ "use_cache": true,
153
+ "vision_lr": null,
154
+ "vocab_size": 32064
155
+ }
non_lora_state_dict.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cd17d61d6c3a6760fa806259c63ccba813340f4166b5ef78dc9c31c035379ded
3
+ size 1333
trainer_state.json ADDED
The diff for this file is too large to render. See raw diff
 
training_log_stage7.txt ADDED
The diff for this file is too large to render. See raw diff
 
training_log_stage7_v2.txt ADDED
The diff for this file is too large to render. See raw diff