Quidest?

goLLuM

Small library for LLM completion. It provides a single client to interface to multiple LLM providers.

LLM providers supported:

Completion

 1package main
 2
 3import (
 4  "bytes"
 5  "encoding/json"
 6  "fmt"
 7  "io"
 8  "time"
 9
10  g "github.com/azr4e1/gollum"
11)
12
13func main() {
14  // create new client
15  cOllama, err := g.NewClient(g.WithProvider(g.OLLAMA), g.WithApiBase("http://localhost:11434"))
16  if err != nil {
17    panic(err)
18  }
19  cOllama.Timeout = 200 * time.Second // default is 30 for http request
20
21  // create a chat
22  chat := g.NewChat(g.SystemMessage("You are a Yakuza member. Act like it! Do not use emoji, and be very straightforward and to the point."), g.UserMessage("What is the difference between Proof of Work and Proof of Stake in Blockchain? What is your opinion on this? Which one is better?"))
23
24  _, res, err := cOllama.Complete(g.WithModel("gemma2:2b"), g.WithMessages(chat.History()))
25  if err != nil {
26    panic(err)
27  }
28
29  b := new(bytes.Buffer)
30  enc := json.NewEncoder(b)
31  enc.SetIndent("", "  ")
32  err = enc.Encode(res)
33  if err != nil {
34  	panic(err)
35  }
36  j, err := io.ReadAll(b)
37  if err != nil {
38  	panic(err)
39  }
40}

Response:

 1{
 2  "Id": "",
 3  "Object": "",
 4  "Created": 38,
 5  "Model": "gemma2:2b",
 6  "Choices": [
 7    {
 8      "Index": 0,
 9      "Message": {
10        "Role": "assistant",
11        "Content": "Proof of work (PoW) requires massive computing power to solve complex equations.  This ensures the network's integrity, as only legitimate participants can contribute.\n\nProof of stake (PoS) uses a system where validators lock up their cryptocurrency as collateral. The more you have, the higher your chance of being selected to create and verify transactions. \n\nMy opinion? PoW is too cumbersome, energy-hungry. It's outdated.  PoS offers efficiency and less environmental impact. This leads to faster network speeds and lower costs. \n\nBetter? PoS is a cleaner solution, more sustainable in the long term. \n"
12      },
13      "FinishReason": "done"
14    }
15  ],
16  "Usage": {
17    "PromptTokens": 66,
18    "CompletionTokens": 131,
19    "TotalTokens": 197,
20    "CompletionTokensDetails": null
21  },
22  "Error": {
23    "Message": "",
24    "Type": ""
25  },
26  "StatusCode": 200
27}

Streaming is also supported. You need to provide a streaming function that will operate on the stream

 1cOllama.EnableStream(func(req g.CompletionResponse) error {
 2  messages := req.Messages()
 3  if len(messages) == 1 {
 4    fmt.Print(messages[0])
 5  }
 6  return nil
 7})
 8_, _, err := cOllama.Complete(g.WithModel("gemma2:2b"), g.WithMessages(chat.History()))
 9if err != nil {
10	panic(err)
11}

Result:

1You want to know about Proof of Work and Proof of Stake, huh?
2
3* **Proof of Work:**  Think of it like a competition. Miner have to solve complex math problems before they can add a new block to the chain. Takes tons of energy and resources.  Old school.
4* **Proof of Stake:**  Instead of racing, you just need to own some coins.  The more you own, the bigger chance you get to propose a new block.  More sustainable.
5
6Personally? I think PoS is cleaner, faster, cheaper. It's the future. Less headache for everyone.  But every coin gotta have its own method.

the text will appear as a stream on your terminal.

Text To Speech

Currently only openai is supported

 1package main
 2
 3import (
 4  "os"
 5  "time"
 6
 7  g "github.com/azr4e1/gollum"
 8)
 9
10func main() {
11  apiKey := os.Getenv("OPENAI_API_KEY")
12  client, err := g.NewClient(g.WithAPIKey(apiKey), g.WithProvider(g.OPENAI))
13  if err != nil {
14    panic(err)
15  }
16  client.Timeout = 200 * time.Second
17  _, res, err := client.TextToSpeech(g.WithTTSInput("My name is Ken Takakura, but my friends call me Okarun."), g.WithTTSVoice("onyx"), g.WithTTSModel("tts-1-hd"))
18  if err != nil {
19    panic(err)
20  }
21
22  audio := res.Audio
23  audioFile, err := os.Create("speech.mp3")
24  defer audioFile.Close()
25  if err != nil {
26    panic(err)
27  }
28  _, err = audioFile.Write(audio)
29  if err != nil {
30    panic(err)
31  }
32}