Update README.md

README.md

@@ -11,12 +11,21 @@ pipeline_tag: text-classification
 ---
 # QA-Evaluation-Metrics
 
-[![PyPI version qa-metrics](https://img.shields.io/pypi/v/qa-metrics.svg)](https://pypi.org/project/qa-metrics/) [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/17b7vrZqH0Yun2AJaOXydYZxr3cw20Ga6?usp=sharing)
+[![PyPI version qa-metrics](https://img.shields.io/pypi/v/qa-metrics.svg)](https://pypi.org/project/qa-metrics/) 
+[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/17b7vrZqH0Yun2AJaOXydYZxr3cw20Ga6?usp=sharing)
 
-QA-Evaluation-Metrics is a fast and lightweight Python package for evaluating question-answering models. It provides various basic metrics to assess the performance of QA models. Check out our paper [**PANDA**](https://arxiv.org/abs/2402.11161), a matching method going beyond token-level matching and is more efficient than LLM matchings but still retains competitive evaluation performance of transformer LLM models.
+QA-Evaluation-Metrics is a fast and lightweight Python package for evaluating question-answering models and prompting of black-box and open-source large language models. It provides various basic metrics to assess the performance of QA models. Check out our paper [**PANDA**](https://arxiv.org/abs/2402.11161), an efficient QA evaluation that retains competitive evaluation performance of transformer LLM models. 
+
+### Updates
+- Uopdated to version 0.2.8 
+  - Supports prompting OPENAI GPT-series models and Claude Series models now. (Assuimg OPENAI version > 1.0)
+  - Supports prompting various open source models such as LLaMA-2-70B-chat, LLaVA-1.5 etc by calling API from [deepinfra](https://deepinfra.com/models).
 
 
 ## Installation
+* Python version >= 3.6
+* openai version >= 1.0
+
 
 To install the package, run the following command:
 
@@ -26,20 +35,7 @@ pip install qa-metrics
 
 ## Usage
 
-The python package currently provides four QA evaluation metrics.
-
-#### Exact Match
-```python
-from qa_metrics.em import em_match
-
-reference_answer = ["The Frog Prince", "The Princess and the Frog"]
-candidate_answer = "The movie \"The Princess and the Frog\" is loosely based off the Brother Grimm's \"Iron Henry\""
-match_result = em_match(reference_answer, candidate_answer)
-print("Exact Match: ", match_result)
-'''
-Exact Match:  False
-'''
-```
+The python package currently provides six QA evaluation methods.
 
 #### Prompting LLM For Evaluation
 
@@ -47,10 +43,11 @@ Note: The prompting function can be used for any prompting purposes.
 
 ###### OpenAI
 ```python
-from qa_metrics.prompt_llm import *
-set_openai_api_key(YOUR_OPENAI_KEY)
+from qa_metrics.prompt_llm import CloseLLM
+model = CloseLLM()
+model.set_openai_api_key(YOUR_OPENAI_KEY)
 prompt = 'question: What is the Capital of France?\nreference: Paris\ncandidate: The capital is Paris\nIs the candidate answer correct based on the question and reference answer? Please only output correct or incorrect.'
-prompt_gpt(prompt=prompt, model_engine='gpt-3.5-turbo', temperature=0.1, max_token=10)
+model.prompt_gpt(prompt=prompt, model_engine='gpt-3.5-turbo', temperature=0.1, max_tokens=10)
 
 '''
 'correct'
@@ -59,14 +56,40 @@ prompt_gpt(prompt=prompt, model_engine='gpt-3.5-turbo', temperature=0.1, max_tok
 
 ###### Anthropic
 ```python
-set_anthropic_api_key(YOUR_OPENAI_KEY)
-prompt_claude(prompt=prompt, model_engine='claude-v1', anthropic_version="2023-06-01", max_tokens_to_sample=100, temperature=0.7)
+model = CloseLLM()
+model.set_anthropic_api_key(YOUR_Anthropic_KEY)
+model.prompt_claude(prompt=prompt, model_engine='claude-v1', anthropic_version="2023-06-01", max_tokens_to_sample=100, temperature=0.7)
 
 '''
 'correct'
 '''
 ```
 
+###### deepinfra (See below for descriptions of more models)
+```python
+from qa_metrics.prompt_open_llm import OpenLLM
+model = OpenLLM()
+model.set_deepinfra_key(YOUR_DEEPINFRA_KEY)
+model.prompt(message=prompt, model_engine='mistralai/Mixtral-8x7B-Instruct-v0.1', temperature=0.1, max_tokens=10)
+
+'''
+'correct'
+'''
+```
+
+#### Exact Match
+```python
+from qa_metrics.em import em_match
+
+reference_answer = ["The Frog Prince", "The Princess and the Frog"]
+candidate_answer = "The movie \"The Princess and the Frog\" is loosely based off the Brother Grimm's \"Iron Henry\""
+match_result = em_match(reference_answer, candidate_answer)
+print("Exact Match: ", match_result)
+'''
+Exact Match:  False
+'''
+```
+
 #### Transformer Match
 Our fine-tuned BERT model is this repository. Our Package also supports downloading and matching directly. distilroberta, distilbert, and roberta are also supported now! 🔥🔥🔥