Skip to content
Back to search

foundation-models-utilities

Emerging and experimental patterns for building with the Foundation Models framework

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/apple--foundation-models-utilities/badge.svg)](/listings/apple--foundation-models-utilities)

HTML

<a href="/listings/apple--foundation-models-utilities"><img src="/api/listings/apple--foundation-models-utilities/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 234 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. <h1>
  2. <img alt="Apple Foundation Models framework logo" src="./assets/fm-icon-27.png" width="90" valign="middle">
  3. &nbsp;Foundation Models framework utilities
  4. </h1>
  5. ## Overview
  6. This package adds extra utilities for working with LLMs via the [Foundation Models framework](https://developer.apple.com/documentation/FoundationModels), such as custom skills, context management helpers, and a chat completions client that connects to a hosted model of your choice.
  7. - 💻 **Supported platforms**: Apple platforms and select Linux distributions like Ubuntu
  8. - 🛠️ **Coding agent skills**: [`skills/`](skills/) can teach your favorite coding agent how to use this package
  9. - 💬 **Issue reporting**: [Apple Developer Forums](https://developer.apple.com/forums/topics/machine-learning-and-ai/machine-learning-and-ai-foundation-models)
  10. ## Use the Package
  11. **Xcode** (Apple platforms)
  12. 1. From the Xcode menu bar, choose File > Add Package Dependencies.
  13. 2. Enter the package URL: https://github.com/apple/foundation-models-utilities.
  14. 3. Xcode downloads the package assets.
  15. **Swift Package Manager** (all supported platforms)
  16. Add as a dependency to your Swift package `Package.swift` file like this:
  17. ```swift
  18. let package = Package(
  19. name: "YourApp",
  20. dependencies: [
  21. .package(url: "https://github.com/apple/foundation-models-utilities", from: "1.0.0-beta6")
  22. ],
  23. targets: [
  24. .target(
  25. name: "YourApp",
  26. dependencies: [
  27. .product(name: "FoundationModelsUtilities", package: "foundation-models-utilities")
  28. ]
  29. )
  30. ]
  31. )
  32. ```
  33. ## Key Features
  34. ### ChatCompletionsLanguageModel
  35. The `ChatCompletionsLanguageModel` can communicate with any server that uses the chat completions REST API.
  36. ```swift
  37. let model = ChatCompletionsLanguageModel(
  38. name: "minimax-m2.5",
  39. url: URL(string: "http://localhost:8000/v1")!,
  40. )
  41. let session = LanguageModelSession(model: model)
  42. let response = try await session.respond(to: "How many folds does it take to make a paper crane?")
  43. print(response.content)
  44. ```
  45. When initializing the model, you can specify some of its capabilities based on the server you're connecting to. Some local LLM servers don't support guided generation, for example.
  46. ```swift
  47. let model = ChatCompletionsLanguageModel(
  48. name: "minimax-m2.5",
  49. url: URL(string: "http://localhost:8000/v1")!,
  50. supportsGuidedGeneration: false
  51. )
  52. ```
  53. The `ChatCompletionsLanguageModel` is especially useful for integrating with a large ecosystem of open source utilities built around the chat completions protocol.
  54. ### History Management
  55. This package also contains a collection of profile modifiers to help you compress a session's transcript and prevent it from outgrowing the model's context window. Strategies include dropping completed tool calls, establishing a rolling window, and summarizing previous interactions into a single entry. There's no one-size-fits-all solution, so we encourage composing strategies together to suit your specific application.
  56. This example combines three history management modifiers. Modifiers apply in outside-in order: first, the profile drops completed tool calls, then applies a rolling window. Summarization runs only if the rolling window of 10 entries exceeds 5000 tokens.
  57. ```swift
  58. struct MyProfile: LanguageModelSession.DynamicProfile {
  59. let status: Status
  60. var body: some DynamicProfile {
  61. Profile {
  62. Instructions("A conversation between a user and a helpful assistant.")
  63. ToggleDarkModeTool()
  64. }
  65. .summarizeHistory(entryThreshold: 10, model: status.summarizerModel)
  66. .rollingWindow(entries: 10)
  67. .droppingCompletedToolCalls()
  68. }
  69. }
  70. ```
  71. ### Skills
  72. `Skills` allow adding extra directions about performing specific tasks into a `LanguageModelSession` transcript on a just-in-time basis. This prevents context pollution and helps optimize time-to-first-token.
  73. The `Skills` type conforms to `DynamicInstructions` and is initialized using a result builder. You must provide a `SkillActivations` instance that tracks which skills are currently active. Because `SkillActivations` conforms to `Observable`, you can use it to drive UI updates.
  74. ```swift
  75. @Observable
  76. class Assistant {
  77. var activations = SkillActivations()
  78. }
  79. struct MyProfile: LanguageModelSession.DynamicProfile {
  80. let assistant: Assistant
  81. var body: some DynamicProfile {
  82. Profile {
  83. Instructions("A conversation between a user and a helpful assistant.")
  84. ToggleDarkModeTool()
  85. Skills(activations: assistant.activations) {
  86. Skill(
  87. name: "style-guide",
  88. description: "Applies the project's writing style guide",
  89. prompt: """
  90. # Style Guide
  91. ## Keep phrasing literal
  92. Idioms and figurative phrases can add color, but they slow
  93. down readers who are scanning, learning the language, or
  94. translating the text. Prefer literal phrasing that names
  95. what you mean, and reserve metaphor for places where it
  96. genuinely earns its keep.
  97. ...(continued)
  98. """
  99. )
  100. Skill(
  101. name: "calendaring",
  102. description: "Read and modify the user's calendar",
  103. instructions: """
  104. Unless specified otherwise, all work meetings
  105. should start 5 minutes after the hour
  106. """
  107. )
  108. }
  109. }
  110. }
  111. }
  112. ```
  113. A `Skill` can be initialized with a `prompt` string (or a trailing `@PromptBuilder`) or with an `instructions` string. The initializer you choose affects where the additional content is added to the transcript.
  114. In both cases, the model activates a skill by generating a tool call.
  115. When initialized with a prompt, a skill's content is added into the transcript as part of a matching tool output. This has the advantage of not invalidating the key-value cache.
  116. ```
  117. Before After
  118. ┌───────────────────────┐ ┌───────────────────────┐
  119. │ Instructions │ │ Instructions │
  120. │ (original) │ │ (original) │
  121. ├───────────────────────┤ ├───────────────────────┤
  122. │ Prompt │ │ Prompt │
  123. └───────────────────────┘ ├───────────────────────┤
  124. │ Tool Call │
  125. │ (activate: skill_a) │
  126. ├───────────────────────┤
  127. │ Tool Output │
  128. │ (skill_a content) │
  129. ├───────────────────────┤
  130. │ Response │
  131. └───────────────────────┘
  132. ```
  133. When initialized with instructions, a skill's content is inserted at the end of the first instructions entry in the transcript. Models are typically trained to obey instructions with high priority, but doing so often comes at the cost of a key-value cache invalidation.
  134. ```
  135. Before After
  136. ┌────────────────────────────────┐ ┌────────────────────────────────┐
  137. │ Instructions │ │ Instructions │
  138. │ (original) │ │ (original + skill_a content) │
  139. ├────────────────────────────────┤ ├────────────────────────────────┤
  140. │ Prompt │ │ Prompt │
  141. └────────────────────────────────┘ ├────────────────────────────────┤
  142. │ Tool Call │
  143. │ (activate: skill_a) │
  144. ├────────────────────────────────┤
  145. │ Tool Output │
  146. │ (skill activated message) │
  147. ├────────────────────────────────┤
  148. │ Response │
  149. └────────────────────────────────┘
  150. ```
  151. Instructions-based skills can optionally be deactivated by the model after activation. Pass `allowsDeactivation: true` to enable this.
  152. ```swift
  153. Skill(
  154. name: "calendaring",
  155. description: "Read and modify the user's calendar",
  156. instructions: """
  157. Unless specified otherwise, all work meetings
  158. should start 5 minutes after the hour
  159. """,
  160. allowsDeactivation: true
  161. )
  162. ```
  163. The model may now issue a second tool call to remove the skill's content from its instructions. This can be a powerful tool for combating context pollution, especially when combined with history transformations that remove complete tool calls.
  164. ```
  165. Before After Dropping Completed Tool Calls
  166. ┌────────────────────────────────┐ ┌────────────────────────────────┐ ┌────────────────────────────────┐
  167. │ Instructions │ │ Instructions │ │ Instructions │
  168. │ (original + skill_a content) │ │ (original) │ │ (original) │
  169. ├────────────────────────────────┤ ├────────────────────────────────┤ ├────────────────────────────────┤
  170. │ Prompt │ │ Prompt │ │ Prompt │
  171. ├────────────────────────────────┤ ├────────────────────────────────┤ ├────────────────────────────────┤
  172. │ Tool Call │ │ Tool Call │ │ Response │
  173. │ (activate: skill_a) │ │ (activate: skill_a) │ ├────────────────────────────────┤
  174. ├────────────────────────────────┤ ├────────────────────────────────┤ │ Prompt │
  175. │ Tool Output │ │ Tool Output │ ├────────────────────────────────┤
  176. │ (skill activated message) │ │ (skill activated message) │ │ Response │
  177. ├────────────────────────────────┤ ├────────────────────────────────┤ └────────────────────────────────┘
  178. │ Response │ │ Response │
  179. └────────────────────────────────┘ ├────────────────────────────────┤
  180. │ Prompt │
  181. ├────────────────────────────────┤
  182. │ Tool Call │
  183. │ (deactivate: skill_a) │
  184. ├────────────────────────────────┤
  185. │ Tool Output │
  186. │ (skill deactivated message) │
  187. ├────────────────────────────────┤
  188. │ Response │
  189. └────────────────────────────────┘
  190. ```

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.