Update README.md

README.md

@@ -59,3 +59,86 @@ model = AutoModelForCausalLM.from_pretrained(
 
 # Input: Extract important facial and hair details from the description and give them in a comma-separated string. Suspect is a female, likely in her late 20s to early 30s, with a distinctly oval-shaped face. She possesses soft, rounded features, including full lips. Her nose is petite and slightly upturned and has almond-shaped eyes of a deep brown color with long, fluttery eyelashes. She has high cheekbones, and her skin is smooth, unblemished. She has expressive eyebrows, that are gently arched.
 # Output: Female, late 20s to early 30s, oval-shaped face, soft rounded features, full lips, petite and slightly upturned nose, almond-shaped deep brown eyes, long fluttery eyelashes, high cheekbones, smooth unblemished skin, expressive gently arched eyebrows.
+```
+
+# Model Fine-tuning and Quantization
+
+This model has been fine-tuned for a specific focus on facial detail extraction. It has been fine-tuned and quantized to 4 bits using AutoTrain's advanced capabilities. Here's the code used for fine-tuning and quantization:
+
+``` python
+# Step 1: Setup Environment
+!pip install pandas autotrain-advanced -q
+!autotrain setup --update-torch
+
+# Step 2: Connect to HuggingFace for Model Upload
+
+from huggingface_hub import notebook_login
+notebook_login()
+
+# Step 3: Upload your dataset
+!mv finetune-llama-2/train.csv train.csv
+
+import pandas as pd
+df = pd.read_csv("train.csv")
+print(df.head())
+
+# Step 4: Overview of AutoTrain command
+!autotrain llm --train --project-name mistral-7b-mj-finetuned \
+  --model alexsherstinsky/Mistral-7B-v0.1-sharded --data-path . \
+  --use-peft --quantization int4 --batch-size 2 --epochs 3 --trainer sft \
+  --target-modules q_proj,v_proj --push-to-hub --username replace_it --token replace_it \
+  --lr 2e-4
+
+# Optionally check the available options
+!autotrain llm --help
+
+# Step 5: Completed 🎉
+# After the command above is completed your Model will be uploaded to Hugging Face.
+
+# Step 6: Inference Engine
+!pip install -q peft accelerate bitsandbytes safetensors
+
+import torch
+from peft import PeftModel
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import transformers
+
+adapters_name = "YasiruDEX/mistral-7b-mj-finetuned-face-feature-extraction"
+model_name = "mistralai/Mistral-7B-Instruct-v0.1"  # or your preferred model
+
+device = "cuda"  # the device to load the model onto
+
+bnb_config = transformers.BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_use_double_quant=True,
+    bnb_4bit_quant_type="nf4",
+    bnb_4bit_compute_dtype=torch.bfloat16
+)
+
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    load_in_4bit=True,
+    torch_dtype=torch.bfloat16,
+    device_map='auto'
+)
+
+# Step 7: Peft Model Loading with upload model
+model = PeftModel.from_pretrained(model, adapters_name)
+
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+tokenizer.bos_token_id = 1
+
+print(f"Successfully loaded the model {model_name} into memory")
+
+text = "[INST] Extract important facial and hair details from the description and give them in a comma separated string. Suspect is a female, likely in her late 20s to early 30s, with a distinctly oval-shaped face. She possesses soft, rounded features, including full lips. Her nose is petite and slightly upturned and has almond-shaped eyes of a deep brown color with long, fluttery eyelashes. She has high cheekbones, and her skin is smooth, unblemished. She has expressive eyebrows, that are gently arched. [/INST]"
+
+encoded = tokenizer(text, return_tensors="pt", add_special_tokens=False)
+model_input = encoded
+model.to(device)
+generated_ids = model.generate(**model_input, max_new_tokens=200, do_sample=True)
+decoded = tokenizer.batch_decode(generated_ids)
+print(decoded[0])
+```
+
+
+This additional information provides insights into the model's training process, including its fine-tuning and quantization steps.