MarineSparkLM-4Bv1

A LoRA instruction adapter for marine and offshore engineering, trained on top of zjunlp/OceanGPT-basic-4B-Instruct (4.06B parameters, Qwen3 architecture). This adapter trains 33,030,144 parameters (0.81% of the base model) and is answer-masked instruction-tuned on 2,295 question-answer pairs across ten marine/offshore engineering themes (naval architecture, hydrodynamics, propulsion and machinery, offshore structures, offshore energy, control and autonomy, marine vehicles, sensors and communications, simulation and modelling, and maritime operations).

On a 20-prompt hand-scored evaluation (accuracy + topicality + specificity, 0-2 each, max 120), this adapter scores 84/120 against 42/120 for the bare base model and 56/120 for a continuation-only adapter. A second evaluation on prompts never seen in training, validation, or checkpoint selection confirms the gain at high significance (paired Wilcoxon p=0.0001). Full methodology, all four evaluation rounds, and stated limitations are in the paper (see Citation below) β€” in particular, absolute accuracy remains modest (14-16 of 40), and this adapter should not be treated as a verified source of engineering fact.

Developed by

Dr Cheng Siong Chin, Faculty of Science, Agriculture and Engineering, Newcastle University (Singapore). Contact: cheng.chin@newcastle.ac.uk

How to use

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "zjunlp/OceanGPT-basic-4B-Instruct"
adapter_id = "mcschin75/MarineSparkLM-4Bv1"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base_model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base_model, adapter_id)

messages = [{"role": "user", "content": "Explain the key considerations in ballast water treatment systems for compliance with maritime regulations."}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
output = model.generate(inputs, max_new_tokens=200, temperature=0.8, top_k=50, repetition_penalty=1.2, no_repeat_ngram_size=3)
print(tokenizer.decode(output[0], skip_special_tokens=True))

4-bit loading (NF4, via bitsandbytes) is also supported and roughly halves load time and memory versus bf16, with comparable output quality β€” see the paper's quantization benchmark.

Training data

2,295 question-answer pairs: 114 hand-authored by reading a real corpus passage and writing a question it answers, plus 2,181 generated by prompting the unadapted base model (to avoid feeding one adapter's biases into the other) from 1,197 further precision-filtered passages, with loss masked to answer tokens only. Full dataset construction, filtering steps, and known limitations (residual topical noise β€” an author-reviewed check of a 30-item eval subset found 4 items off-topic, 13%, higher than the earlier low-single-digit estimate, see the paper's Round 5 β€” licence variation across sources) are documented in the paper.

Limitations

  • Absolute accuracy is modest; the adapter biases the model toward the marine-engineering domain and toward committing to a concrete answer, but does not guarantee factual correctness.
  • Reference answers used for evaluation were written by the base model itself. The paper's Round 5 checks this directly on a 30-item subset with author-written references instead and finds the central comparisons hold, but that check covers a quarter of the enlarged eval set with a single reviewer, not the full set.
  • Retrieval-augmented generation on top of this adapter shows no significant benefit in the paper's held-out evaluations.
  • Outputs should be checked independently before any engineering, safety, or regulatory decision.

Citation

If you use this adapter, please cite:

@misc{chin2026marinesparklm,
  title         = {MarineSparkLM-4B: A Lightweight Language Model for Marine and Offshore Engineering Built by Parameter-Efficient Adaptation},
  author        = {Chin, Cheng Siong},
  year          = {2026},
  eprint        = {2611.19151},
  archivePrefix = {arXiv},
  doi           = {10.48550/arXiv.2611.19151},
  url           = {https://doi.org/10.48550/arXiv.2611.19151}
}

Framework versions

  • PEFT 0.21.2
Downloads last month
35
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for mcschin75/MarineSparkLM-4Bv1

Adapter
(1)
this model

Paper for mcschin75/MarineSparkLM-4Bv1