Qwen3-4B-Instruct-2507-satirical-headlines-CoT

This is a fine-tuned version of Qwen/Qwen3-4B-Instruct-2507 designed to generate satirical news headlines in the style of The Onion.

This model was fine-tuned using a synthetic Chain-of-Thought (CoT) dataset. Google's Gemini 2.5 Flash select as a teacher model to generate a CoT analysis for each headline, determining its subject and satirical angle. This approach allows the model to learn the underlying reasoning behind satire, not just the writing style.

This model is the final result of the project detailed in this GitHub repository: Satire On Demand.

Quickstart

The code supporting Qwen3 was recently added to transformers. Ensure you have the latest versions of transformers and peft installed.

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_name = "Qwen/Qwen3-4B-Instruct-2507"
model_name = "joshuachin/Qwen3-4B-Instruct-2507-satirical-headlines-CoT"

# Load the tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    torch_dtype="auto",
    device_map="auto"
)
model = PeftModel.from_pretrained(model, model_name)
model.eval()

# prepare the model input
prompt = "Give me a satirical headline."
messages = [
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

# conduct text completion
generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=1024
)
output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist() 

content = tokenizer.decode(output_ids, skip_special_tokens=True)

print("headline: ", content)

Model Description

• Base Model: Qwen/Qwen3-4B-Instruct-2507
• Training Method: QLoRA fine-tuning on a single GPU.
• Training Library: Unsloth, for significant speed and memory optimization.
• Training Data: A synthetic dataset of 240 examples. Each example consists of a real The Onion headline paired with a "Chain-of-Thought" analysis (Subject + Satirical Angle) generated by gemini-2.5-flash. The model was trained to generate the final headline after seeing the CoT analysis, teaching it to mimic the reasoning process.

Limitations & Bias

This is an experimental model and has several key limitations:

1. Topicality: The model was trained on a dataset of headlines published before 2023. Its references will often feel dated.
2. Inherited Bias: The model inherits the biases of both its base model (Qwen3), and especially of its training data (The Onion). The humor reflects a specific, often American-centric, progressive-leaning worldview and may not be appropriate or funny in all contexts.

Citation

This model was fine-tuned by Joshua Chin. If you use this model, please cite the original creators and data sources:

• Base Model: The Qwen Team (https://github.com/QwenLM/Qwen3)
• Satirical Data: The Onion (https://www.theonion.com)