Add AuK Base and AuK-Flash GGUF components

.gitattributes

@@ -33,3 +33,45 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+auk-base-f16.gguf filter=lfs diff=lfs merge=lfs -text
+auk-base-f32.gguf filter=lfs diff=lfs merge=lfs -text
+auk-base-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+auk-flash-f16.gguf filter=lfs diff=lfs merge=lfs -text
+auk-flash-f32.gguf filter=lfs diff=lfs merge=lfs -text
+auk-flash-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+auk-vae-f32.gguf filter=lfs diff=lfs merge=lfs -text
+examples/cpp/add_cough.wav filter=lfs diff=lfs merge=lfs -text
+examples/cpp/deep_voice.wav filter=lfs diff=lfs merge=lfs -text
+examples/cpp/enhance.wav filter=lfs diff=lfs merge=lfs -text
+examples/cpp/get_what_speaker.wav filter=lfs diff=lfs merge=lfs -text
+examples/cpp/happy.wav filter=lfs diff=lfs merge=lfs -text
+examples/cpp/increase_volume.wav filter=lfs diff=lfs merge=lfs -text
+examples/cpp/instruct.wav filter=lfs diff=lfs merge=lfs -text
+examples/cpp/raise_pitch.wav filter=lfs diff=lfs merge=lfs -text
+examples/cpp/replace_lyrics.wav filter=lfs diff=lfs merge=lfs -text
+examples/cpp/replace_words.wav filter=lfs diff=lfs merge=lfs -text
+examples/cpp/second_speaker.wav filter=lfs diff=lfs merge=lfs -text
+examples/cpp/speed_1_5x.wav filter=lfs diff=lfs merge=lfs -text
+examples/cpp/standard_mandarin.wav filter=lfs diff=lfs merge=lfs -text
+examples/cpp/vocals_only.wav filter=lfs diff=lfs merge=lfs -text
+examples/cpp/whisper.wav filter=lfs diff=lfs merge=lfs -text
+examples/cpp/zero_shot.wav filter=lfs diff=lfs merge=lfs -text
+examples/python/add_cough.wav filter=lfs diff=lfs merge=lfs -text
+examples/python/deep_voice.wav filter=lfs diff=lfs merge=lfs -text
+examples/python/enhance.wav filter=lfs diff=lfs merge=lfs -text
+examples/python/get_what_speaker.wav filter=lfs diff=lfs merge=lfs -text
+examples/python/happy.wav filter=lfs diff=lfs merge=lfs -text
+examples/python/increase_volume.wav filter=lfs diff=lfs merge=lfs -text
+examples/python/instruct.wav filter=lfs diff=lfs merge=lfs -text
+examples/python/raise_pitch.wav filter=lfs diff=lfs merge=lfs -text
+examples/python/replace_lyrics.wav filter=lfs diff=lfs merge=lfs -text
+examples/python/replace_words.wav filter=lfs diff=lfs merge=lfs -text
+examples/python/second_speaker.wav filter=lfs diff=lfs merge=lfs -text
+examples/python/speed_1_5x.wav filter=lfs diff=lfs merge=lfs -text
+examples/python/standard_mandarin.wav filter=lfs diff=lfs merge=lfs -text
+examples/python/vocals_only.wav filter=lfs diff=lfs merge=lfs -text
+examples/python/whisper.wav filter=lfs diff=lfs merge=lfs -text
+examples/python/zero_shot.wav filter=lfs diff=lfs merge=lfs -text
+qwen2.5-omni-3b-bf16.gguf filter=lfs diff=lfs merge=lfs -text
+qwen2.5-omni-3b-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+tokenizer/tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,88 @@
+---
+license: mit
+base_model: tencent/AuK
+pipeline_tag: text-to-speech
+tags:
+  - audio
+  - speech
+  - text-to-speech
+  - zero-shot-tts
+  - voice-cloning
+  - speech-generation
+  - speech-editing
+  - speech-enhancement
+  - source-separation
+  - speech-separation
+  - instruction-guided
+  - diffusion
+  - audio.cpp
+---
+
+# AuK GGUF for audio.cpp
+
+These GGUF files are for use with [audio.cpp](https://github.com/0xShug0/audio.cpp). The card's `base_model` points to the [Tencent AuK base model](https://huggingface.co/tencent/AuK). This package also includes optional files converted from [AuK-Flash](https://huggingface.co/tencent/AuK-Flash); Flash is a separate distilled variant and is not the pinned base model.
+
+> **Output quality:** We validate parity with upstream Python for the tested configuration, not whether AuK's output meets every quality expectation. If a result is disappointing, listen to the corresponding Python reference WAV linked below first. A similar Python result points to the upstream model's behavior, not necessarily an audio.cpp conversion issue. The quantized component combinations have not received the same 16-task parity validation.
+
+Pass the downloaded directory to `audiocpp_cli --model`. Keep the `config/` and `tokenizer/` directories alongside the GGUF files. The default components are `auk-base-f32.gguf`, `qwen2.5-omni-3b-bf16.gguf`, and `auk-vae-f32.gguf`. AuK-Flash requires `--session-option auk.variant=flash`; its default generator is `auk-flash-f32.gguf`. The Qwen component is Qwen2.5-Omni-3B.
+
+For example, instruction TTS with explicitly selected Base components:
+
+```bash
+audiocpp_cli --task tts --family auk \
+  --model /path/to/AuK-Base-and-Flash-GGUF --backend cuda \
+  --session-option auk.variant=base \
+  --session-option auk.model_gguf=auk-base-f32.gguf \
+  --session-option auk.qwen_gguf=qwen2.5-omni-3b-bf16.gguf \
+  --session-option auk.vae_gguf=auk-vae-f32.gguf \
+  --text "The next train leaves in ten minutes." \
+  --request-option "instruct=A warm male voice speaking clear English." \
+  --request-option duration_sec=4 --seed 42 --log --out speech.wav
+```
+
+To use AuK-Flash, select `auk.variant=flash` and an `auk-flash-*.gguf` generator. To use the Qwen Q8_0 component, select `auk.qwen_gguf=qwen2.5-omni-3b-q8_0.gguf`. Keep the F32 VAE in either case.
+
+For editing, use `--task gen --audio input.wav --text "<editing instruction>"`. See the [audio.cpp AuK guide](https://github.com/0xShug0/audio.cpp/blob/main/docs/models/auk.md) and the [AuK cookbook](https://github.com/Tencent-Hunyuan/AuK/blob/main/docs/COOKBOOK.md) for request options and example instructions.
+
+## Sixteen-task validation
+
+The 16-task C++/Python comparison forced FP32 inference and disabled TF32 in both implementations. Input audio and reference voices came from the [upstream AuK demo assets](https://github.com/Tencent-Hunyuan/AuK/tree/main/assets/demo-input-audio). Durations not shown below matched the source recording.
+
+All 16 C++ requests completed and produced 24 kHz WAVs with the same frame counts as the Python outputs. The cosine values compare C++ WAVs with the Python baseline. Waveform cosine alone does not establish whether an edit followed its instruction.
+
+| Task | Tested instruction / setting | C++ vs Python WAV cosine | Listen |
+| --- | --- | ---: | --- |
+| Zero-shot TTS | Reference voice; target sentence; 6 s | 0.999999992 | [Python](./examples/python/zero_shot.wav) / [C++](./examples/cpp/zero_shot.wav) |
+| Instruct TTS | Calm woman speaking clear English; 3 s | 0.999999898 | [Python](./examples/python/instruct.wav) / [C++](./examples/cpp/instruct.wav) |
+| Speech content editing | Replace "but accepting what we cannot have" with "and living well with dreams unmet"; 7 s | 0.999999982 | [Python](./examples/python/replace_words.wav) / [C++](./examples/cpp/replace_words.wav) |
+| Lyric editing | Replace "rear view" with "like you" in isolated vocals | 1.000000000 | [Python](./examples/python/replace_lyrics.wav) / [C++](./examples/cpp/replace_lyrics.wav) |
+| Pitch editing | Raise pitch by 2 semitones | 0.999989991 | [Python](./examples/python/raise_pitch.wav) / [C++](./examples/cpp/raise_pitch.wav) |
+| Speed editing | Speed 1.5x; 6.86 s | 0.999999993 | [Python](./examples/python/speed_1_5x.wav) / [C++](./examples/cpp/speed_1_5x.wav) |
+| Volume editing | Increase volume by 10 dB | 0.999999997 | [Python](./examples/python/increase_volume.wav) / [C++](./examples/cpp/increase_volume.wav) |
+| Emotion editing | Change emotion to happy | 0.999999993 | [Python](./examples/python/happy.wav) / [C++](./examples/cpp/happy.wav) |
+| Timbre editing | Change to a deep, calm male voice | 0.999999991 | [Python](./examples/python/deep_voice.wav) / [C++](./examples/cpp/deep_voice.wav) |
+| De-accent | Change Sichuan-accented speech to standard Mandarin | 0.999999795 | [Python](./examples/python/standard_mandarin.wav) / [C++](./examples/cpp/standard_mandarin.wav) |
+| Nonverbal editing | Add a cough before "We tested"; 10.44 s | 0.999991188 | [Python](./examples/python/add_cough.wav) / [C++](./examples/cpp/add_cough.wav) |
+| Whisper conversion | Speak the source in a quiet whisper | 0.999998371 | [Python](./examples/python/whisper.wav) / [C++](./examples/cpp/whisper.wav) |
+| Speech enhancement | Preserve speakers; remove noise and reverberation | 0.999999994 | [Python](./examples/python/enhance.wav) / [C++](./examples/cpp/enhance.wav) |
+| Speech separation | Keep the second speaker to start talking | 0.999999985 | [Python](./examples/python/second_speaker.wav) / [C++](./examples/cpp/second_speaker.wav) |
+| Music separation | Keep the singing voice; remove other audio | 0.999999998 | [Python](./examples/python/vocals_only.wav) / [C++](./examples/cpp/vocals_only.wav) |
+| Target speaker extraction | Keep the speaker saying "get what" | 0.999999997 | [Python](./examples/python/get_what_speaker.wav) / [C++](./examples/cpp/get_what_speaker.wav) |
+
+The no-TF32 C++ run matched the Python WAVs at waveform cosine 0.999989991 or higher for all 16 tasks; the files were not byte-identical. These historical results use the original combined GGUF and are **not** a 16-task parity claim for the component GGUFs in this package.
+
+## Dtypes and validation scope
+
+All component smoke tests used CUDA and the F32 VAE. A pass here means the listed task loaded the selected GGUFs and generated a WAV; it does not establish Python parity or task quality.
+
+| Generator | Qwen conditioner | Tested task(s) |
+| --- | --- | --- |
+| Base F32 | BF16 | Zero-shot TTS; instruct TTS |
+| Base F16 or Q8_0 | BF16 | Instruct TTS, one run per dtype |
+| Base F32 or Q8_0 | Q8_0 | Zero-shot TTS, one run per combination |
+| AuK-Flash F32 | BF16 | Zero-shot TTS; instruct TTS |
+| AuK-Flash F16 or Q8_0 | BF16 | Instruct TTS, one run per dtype |
+| AuK-Flash F32 | Q8_0 | Zero-shot TTS, one run |
+| AuK-Flash F16 | Q8_0 | Instruct TTS, one run |
+
+The Base F16 + Qwen Q8_0 and AuK-Flash Q8_0 + Qwen Q8_0 combinations were **not tested**. Full 16-task Python parity is untested for every component combination, including the default Base F32 and AuK-Flash F32 paths. The VAE is supplied only as F32; no F16 or quantized VAE is supplied or tested. CPU and Vulkan parity are also untested.

auk-base-f16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d7efc034ef18aa6d5080cb8a3a67731df31aeae7f73d68b19400503db89bd586
+size 3061149120

auk-base-f32.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a0ab5bffe8db0c8a9b54ec357ec8a4bd3d3d991a796859d4b9ad55e2b17af93b
+size 6122226368

auk-base-q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dcf96485cf4d44f8dd84eb190348aa3354117380c95e9b3d90b1f4c797503a39
+size 1639358080

auk-flash-f16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f9ad537dbe1f207943a42ee48dad070fb91b62c7b812e0a124f16dfba72dc092
+size 3061149120

auk-flash-f32.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c1592b6aa12eeb4155e060c21efa9caf681e7c0cfd1afc45a11907d69dc0c55
+size 6122226368

auk-flash-q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec6aab06181307a4e7ba9e22d21532338d9f673162e8933c4cc2d9d6dac45e33
+size 1639358080

auk-vae-f32.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:401e96baa1ac958d66fbe2c8cf5d25105c995c46dd46247c879e6114ef503abd
+size 637364224

config/auk-base.yaml

@@ -0,0 +1,45 @@
+model:
+  name: AuK
+  backbone: Flux2Edit
+  cfm_class: CFMEdit
+
+  schedule:
+    t_sampling: logistic_normal
+    P_mean: -0.8
+    P_std: 0.8
+
+  text_encoder:
+    text_encoder_path: ckpts/Qwen2.5-Omni-3B
+
+  arch:
+    dim: 1536
+    heads: 24
+    ff_mult: 2
+    text_hidden_dim: 2048
+    attn_backend: flash_attn # CLI overrides to "torch" for inference (no flash_attn dep)
+    attn_mask_enabled: True
+    checkpoint_activations: True # gradient checkpointing: peak ~91G→~75G at every_n=4
+    checkpoint_every_n_layers: 4 # checkpoint every 4th layer (fewer recomputes, faster than every=1)
+    num_layers: 10
+    num_single_layers: 20
+
+  vae:
+    vae_name: BigVGANFlowVAE
+    vae_model_path: ckpts/AuK/vae.safetensors
+    latent_dim: 64
+    downsample_rate: 480
+    target_sample_rate: 24000
+    model_init_kwargs:
+      upsample_rates: [5, 4, 3, 2, 2, 2]
+      upsample_kernel_sizes: [10, 8, 6, 4, 4, 4]
+      upsample_initial_channel: 1536
+      resblock_kernel_sizes: [3, 7, 11]
+      resblock_dilation_sizes: [[1, 3, 5], [1, 3, 5], [1, 3, 5]]
+      downsample_rates: [2, 2, 2, 3, 4, 5]
+      downsample_channels: [12, 24, 48, 96, 192, 384, 768]
+      snake_logscale: true
+      latent_dim: ${model.vae.latent_dim}
+      use_vae: true
+      causal: true
+      flow_hidden_channels: 256
+      act_causal: true

config/auk-flash.yaml

@@ -0,0 +1,45 @@
+model:
+  name: AuK-Flash
+  backbone: Flux2Edit
+  cfm_class: CFMEdit
+
+  schedule:
+    t_sampling: logistic_normal
+    P_mean: -0.8
+    P_std: 0.8
+
+  text_encoder:
+    text_encoder_path: ckpts/Qwen2.5-Omni-3B
+
+  arch:
+    dim: 1536
+    heads: 24
+    ff_mult: 2
+    text_hidden_dim: 2048
+    attn_backend: flash_attn # CLI overrides to "torch" for inference (no flash_attn dep)
+    attn_mask_enabled: True
+    checkpoint_activations: True # gradient checkpointing: peak ~91G→~75G at every_n=4
+    checkpoint_every_n_layers: 4 # checkpoint every 4th layer (fewer recomputes, faster than every=1)
+    num_layers: 10
+    num_single_layers: 20
+
+  vae:
+    vae_name: BigVGANFlowVAE
+    vae_model_path: ckpts/AuK-Flash/vae.safetensors
+    latent_dim: 64
+    downsample_rate: 480
+    target_sample_rate: 24000
+    model_init_kwargs:
+      upsample_rates: [5, 4, 3, 2, 2, 2]
+      upsample_kernel_sizes: [10, 8, 6, 4, 4, 4]
+      upsample_initial_channel: 1536
+      resblock_kernel_sizes: [3, 7, 11]
+      resblock_dilation_sizes: [[1, 3, 5], [1, 3, 5], [1, 3, 5]]
+      downsample_rates: [2, 2, 2, 3, 4, 5]
+      downsample_channels: [12, 24, 48, 96, 192, 384, 768]
+      snake_logscale: true
+      latent_dim: ${model.vae.latent_dim}
+      use_vae: true
+      causal: true
+      flow_hidden_channels: 256
+      act_causal: true

examples/cpp/add_cough.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3a1ca0a5f7acf99c08b62aba389e31367ec24a03bf4414769e90a7e18825bed4
+size 501164

examples/cpp/deep_voice.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:18b434356c751d94709c29d7289d01e160025c975e0a6c75aa1f1d73fbf08a8a
+size 316844

examples/cpp/enhance.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f4e1d9bbf8b26ca4915ac897759041bfa85b8d964b3767c62236812fe7567df
+size 239084

examples/cpp/get_what_speaker.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:59f9ad9f3616130f3d5a9023afce399d815a0a9a1bf33bf46708d2420a7aeae6
+size 1344044

examples/cpp/happy.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9489b4a2b959d1df45fac4124da5a6495a6bfe5bf463091be5783893669add3f
+size 303404

examples/cpp/increase_volume.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6ea7cf7d76bde0214eedbe4e6faa4cfeb15c7afb484392add730985090d32287
+size 264044

examples/cpp/instruct.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:97bd692a509a5add0924961d4ce23d615002323b509a66e1a49c8485a2aa4c80
+size 144044

examples/cpp/raise_pitch.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2cd2d7c90531bc652af4e1f6fe08b9fe078f70bbe8ac69d0a43ef57554e43b59
+size 264044

examples/cpp/replace_lyrics.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b9a21df14b84c439030ee437c7c865f3b0d8aa98f91f36ab0f2e4a8fe64cc238
+size 275564

examples/cpp/replace_words.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d82a964b77e2e13a4196e1897a79b0b4e6aa1ac687380424f40d198c2b7b7e9d
+size 336044

examples/cpp/second_speaker.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:172b3fb42ad3cb48c3cf535b9e8dfd01bfa61aea1051db4b93288716d5bd66fe
+size 922604

examples/cpp/speed_1_5x.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:07397bf90ae1db2bd2b8f451aabf711cdcdafc96d20a2454b9e7da052830f39b
+size 329324

examples/cpp/standard_mandarin.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2de25af9e8a649ca51daca455826659b8648919f8e4a7c96bf7c9ce0e840efbb
+size 393644

examples/cpp/vocals_only.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:95f32d7823a13a5346e46c7a84c97e4d6b5497ab34c4c8ad3a82bfc551873d85
+size 522284

examples/cpp/whisper.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fdf90cec169e2bf9067baafd918bd24996efb9897484f1dfc1438d916e77f804
+size 411884

examples/cpp/zero_shot.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c357a82f2f770492f9acc430559a886f585b959d119f9b8279f839898c5b7c0
+size 288044

examples/python/add_cough.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a245f085ad8da2982009b9ca617e08539b72de2785ed0138b3a051d0d6d62c55
+size 501198

examples/python/deep_voice.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b2d6f4ec9a9f8922968636df8d8371e840afbdc0f2a9139b08e997107bc3bb41
+size 316878

examples/python/enhance.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:acac796f51c435d6f45e4bc05ec552c583425994fbd20b315922d2ae2ea0d4a1
+size 239118

examples/python/get_what_speaker.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fc31a22582c4f9f0a0573959f0dd3526052de6df60875ecc6e962acd04777e3
+size 1344078

examples/python/happy.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5de91d4090d5998addb0fee97d8905561acd6ef13035b189e859e071aa9422c
+size 303438

examples/python/increase_volume.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3feb4e0134dce0356fcd547d1e3f1f68cb8c2486879ab69076a69999a31cd00b
+size 264078

examples/python/instruct.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b2bf530fbdc0ed78f76c7c0d457e28090ef193dfbad47e372fc5fa7da3628255
+size 144078

examples/python/raise_pitch.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b43bb7cc05cd69056c44689a5c5bf04afb23c4736f4edfccc03e2fdf3aa2c547
+size 264078

examples/python/replace_lyrics.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3029ad8e4cf268fd0cadf6d4e930cf9b2417b6ff8de3ab531b6bba958c9b60e9
+size 275598

examples/python/replace_words.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c9d7ad26e4d8879864b7e6fac386d1975f40d80501430a7476ed7fa898135a37
+size 336078

examples/python/second_speaker.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:68a50244bdbe4936bcdb20fa14fc12c1ec131ff723d79ed7c1a46205719ba3d7
+size 922638

examples/python/speed_1_5x.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8caa3db0dcd1936709fd527c39e645b539a4c0b65d781d679257d9c3face5166
+size 329358

examples/python/standard_mandarin.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f9f39ca2a06a5c6b2e8cf0e074e3f68fd8c37a067006d2bbe862f5bfca72589
+size 393678

examples/python/vocals_only.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:283d10c3b126353b1dac81bffd460c9c99e7106627dc5ade84748c9b369c4a3f
+size 522318

examples/python/whisper.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:55c8f1ee9bd3bd5cf797a065fec5158342c41b929dc13ec9ce8d4251994878d5
+size 411918

examples/python/zero_shot.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:30a403cac5b0cba2ae27d80c89a9a439554df701d8ef65046e26eb27705d2e6b
+size 288078

qwen2.5-omni-3b-bf16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d1807b5849ac7b814c78b68ad4571aa181aa59280ef69dee222e5f6bb834368d
+size 7447373760

qwen2.5-omni-3b-q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:732861a7d938d08770f98d4732ee32982ee042c6c7161eed48db5569a8a620c3
+size 4253964480

tokenizer/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8441917e39ae0244e06d704b95b3124795cec478e297f9afac39ba670d7e9d99
+size 11421870

tokenizer/tokenizer_config.json

@@ -0,0 +1,223 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|AUDIO|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|audio_bos|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|audio_eos|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_bos|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_eos|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|IMAGE|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|VIDEO|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|AUDIO|>",
+    "<|audio_bos|>",
+    "<|audio_eos|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_bos|>",
+    "<|vision_eos|>",
+    "<|vision_pad|>",
+    "<|IMAGE|>",
+    "<|VIDEO|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% set audio_count = namespace(value=0) %}{% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n{% endif %}<|im_start|>{{ message['role'] }}\n{% if message['content'] is string %}{{ message['content'] }}<|im_end|>\n{% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_bos|><|IMAGE|><|vision_eos|>{% elif content['type'] == 'audio' or 'audio' in content or 'audio_url' in content %}{% set audio_count.value = audio_count.value + 1 %}{% if add_audio_id %}Audio {{ audio_count.value }}: {% endif %}<|audio_bos|><|AUDIO|><|audio_eos|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_bos|><|VIDEO|><|vision_eos|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>\n{% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant\n{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {
+    "image_token": "<|IMAGE|>",
+    "audio_token": "<|AUDIO|>",
+    "video_token": "<|VIDEO|>",
+    "vision_bos_token": "<|vision_bos|>",
+    "vision_eos_token": "<|vision_eos|>",
+    "audio_bos_token": "<|audio_bos|>",
+    "audio_eos_token": "<|audio_eos|>"
+  },
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "processor_class": "Qwen2_5OmniProcessor",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "image_token": "<|IMAGE|>",
+  "audio_token": "<|AUDIO|>",
+  "video_token": "<|VIDEO|>",
+  "vision_bos_token": "<|vision_bos|>",
+  "vision_eos_token": "<|vision_eos|>",
+  "audio_bos_token": "<|audio_bos|>",
+  "audio_eos_token": "<|audio_eos|>"
+}