bert-base-uncased_SWAG (RKNN2)

This is an RKNN-compatible version of the ehdwns1516/bert-base-uncased_SWAG model. It has been optimized for Rockchip NPUs using the rk-transformers library.

Click to see the RKNN model details and usage examples

Model Details

• Original Model: ehdwns1516/bert-base-uncased_SWAG
• Target Platform: rk3588
• rknn-toolkit2 Version: 2.3.2
• rk-transformers Version: 0.3.0

Available Model Files

Model File	Optimization Level	Quantization	File Size
model.rknn	0	float16	235.4 MB

Usage

Installation

Install rk-transformers with inference dependencies to use this model:

pip install rk-transformers[inference]

RK-Transformers API

import numpy as np
from rktransformers import RKModelForMultipleChoice
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("rk-transformers/bert-base-uncased_SWAG")
model = RKModelForMultipleChoice.from_pretrained(
    "rk-transformers/bert-base-uncased_SWAG",
    platform="rk3588",
    core_mask="auto",
)

prompt = "In Italy, pizza is served in slices."
choice0 = "It is eaten with a fork and knife."
choice1 = "It is eaten while held in the hand."
choice2 = "It is blended into a smoothie."
choice3 = "It is folded into a taco."

encoding = tokenizer(
    [prompt, prompt, prompt, prompt], [choice0, choice1, choice2, choice3], return_tensors="np", padding=True
)
inputs = {k: np.expand_dims(v, 0) for k, v in encoding.items()}

outputs = model(**inputs)
logits = outputs.logits
print(logits.shape)

Configuration

The full configuration for all exported RKNN models is available in the config.json file.

---

ehdwns1516/bert-base-uncased_SWAG

• This model has been trained as a SWAG dataset.

• Sentence Inference Multiple Choice DEMO: Ainize DEMO

• Sentence Inference Multiple Choice API: Ainize API

Overview

Language model: bert-base-uncased

Language: English

Training data: SWAG dataset

Code: See Ainize Workspace

Usage

In Transformers

from transformers import AutoTokenizer, AutoModelForMultipleChoice
  
tokenizer = AutoTokenizer.from_pretrained("ehdwns1516/bert-base-uncased_SWAG")

model = AutoModelForMultipleChoice.from_pretrained("ehdwns1516/bert-base-uncased_SWAG")

def run_model(candicates_count, context: str, candicates: list[str]):
    assert len(candicates) == candicates_count, "you need " + candicates_count + " candidates"
    choices_inputs = []
    for c in candicates:
        text_a = ""  # empty context
        text_b = context + " " + c
        inputs = tokenizer(
            text_a,
            text_b,
            add_special_tokens=True,
            max_length=128,
            padding="max_length",
            truncation=True,
            return_overflowing_tokens=True,
        )
        choices_inputs.append(inputs)

    input_ids = torch.LongTensor([x["input_ids"] for x in choices_inputs])
    output = model(input_ids=input_ids)

    return {"result": candicates[torch.argmax(output.logits).item()]}

items = list()
count = 4 # candicates count
context = "your context"
for i in range(int(count)):
    items.append("sentence")

result = run_model(count, context, items)