2025-02-19 Wed

• #AI/Coding
  • #Idea
    • MCP/Idea/MCP Metatool
      • MCP/Idea/MCP Metatool

        • 
        • Idea for a "Metatool" to generate Model Context Protocol servers, given specifications of existing tools, in order to be able to consume them in CursorAI, GitHub/Actions or in CLI prompting
  • #Read
    • Person/Harper Reed/Blog/25/02/My LLM codegen workflow atm
      • • Awesome overview of a CLI-centric workflow for AI coding. solid advice.
          • Tools: mise, aider, llm-cli, Repomix added to AI/Coding/Todos
  • #Tool
    • llm-cli - Person/Simon Willison's CLI #LLM tool. Been meaning to try this out for a while.
    • Repomix

      • 📦 Repomix (formerly Repopack) is a powerful tool that packs your entire repository into a single, AI-friendly file. Perfect for when you need to feed your codebase to Large Language Models #LLMs or other AI tools like Claude, ChatGPT, DeepSeek, Perplexity, Gemini, Gemma, Llama, Grok, and more.
        

• AI Evals
  • LangChain/Academy/LangSmith/M4 Collecting Feedback/L2 Annotation Queues
    • • #Q when should we encourage SMEs to add to a distinct LangSmith/Dataset vs when to apply a #Rubric ?
        • The annotation queue has a "D" hotkey for adding to the default dataset, so it might be simplest if we were going to only use datasets for the "golden" dataset - the "reference" of a good response.

    • • Creating LangSmith/Annotation/Rubric

        • we can leave high level instructions, e.g. "Please only score fields you are comfortable with"
        • full list of feedback tags from our LangSmith/Workspace
          • See also
            • Set up feedback criteria | 🦜️🛠️ LangSmith
              • https://smith.langchain.com/settings/workspaces/feedbacks -> link to edit feedback for workspace
            • Feedback data format | 🦜️🛠️ LangSmith
              • this is the data format for each "feedback item"
                • continuous
                • categorical
                  • could be yes/no (binary)
        • For each field we can define special instructions specifically in the context of this annotation #Cool
  • LangChain/Academy/LangSmith/M5 Production Observability
    • Resources for mod 5
  • LangChain/Academy/LangSmith/M5 Production Observability/L1 Filtering

    • • common LangSmith/Filters

        • status = error (all runs with errors)
        • feedback score < x (received bad feedback)
        • run_type = "LLM" and latency > X (slow LLM runs)
        • Metadata["ls_model_name"] = X (create a filter for metadata field)
  • LangChain/Academy/LangSmith/M5 Production Observability/L2 Online Evaluation
    • • Takeaways
        • Langsmith Online Evaluators provide a way in the langsmith UI to define a prompt and a Structured Output schema, and ensure that every time a trace comes in that matches a LangSmith/Filter, it gets classified
          • common use-cases
            • is_hallucinating yes or no
            • helpfulness 1-10
• #Slack
  • Browser/Extension
    • Person/Kazuki Yamada/GitHub/slack-channels-grouping #Hierarchy
      • Slack channels grouping image
• #Tool
  • Person/Jeff Dickey
    • #CLI
      • #oclif - #Salesforce built this Open CLI Framework
    • mise

      • • mise/About

          • mise (pronounced "meez") or mise-en-place is a development environment setup tool.
            • name refers to French culinary phrase roughly translating to "setup" or "put in place"
              • before one begins cooking, a professional chef should have all their utensils and ingredients ready to go and in their place

          • Key Ideas for mise

            • Declar/at/ive configuration

              • The main idea of mise is that mise/Config/mise.toml can let either a project or a user declare the
              • mise.toml describes consistent way to setup and interact with projects across languages
              • mise.toml is a file type used by the mise cli that lets developers globally declare or individual projects locally declare the programming languages, the package manager packages, the environment variables, and the common tasks that need to be run

            • Package/Manager/Meta

              • Mise integrates with multiple Meta Package Managers or Universal Installers. In that sense, it's a meta-meta-package-manager (LOL).

          • The Three Core Jobs to be Done or Main Use Cases for mise

            • mise/JTBD/1 - Dev Tool Installer

              • Job/to be Done 1: as a Declar/at/ive mise Dev Tool installer of Programming Languages and Package Managers packages
                • installs versions of NodeJS, Py, tf as well as any package from those ecosystems
                • specify versions of tools per project or globally
                • mise supports hundreds of dev tools
                • Person/Shunsuke Suzuki created aqua as a more secure Meta Package Manager; mise uses it as a default backend for verified OSS binaries
                • mise/JTBD/1 - Dev Tool Installer/Card/How do I upgrade my global mise Tools

                  • How do I upgrade my global mise Tools

                    • mise mise/JTBD/1 - Dev Tool Installer
                      • How do I upgrade my global mise/Tools? Card
                        • Just use mise/upgrade.
                        • My Notes
                          • AFAIK there's no dedicated global upgrade subcommand that just upgrades global mise tools, but there might be a way to call a command only on a particular (global) mise/Config/mise.toml
                          • Initially I thought you had to be in a directory that didn't have a local mise/Config/mise.toml to upgrade just the global tools, but then I asked mise/Q/Will calling mise upgrade in a directory that has a local mise.toml upgrade global mise Tools? and realized that it does actually upgrade global dependencies even if you are in a directory with a local mise config.
            • mise/JTBD/2 - Environment Manager

              • mise Job/to be Done 2: as a Declar/at/ive environment manager - mise/Environment
                • manages EnvVars for project or globally
                • lets you specify configuration like AWS_ACCESS_KEY_ID that may differ between projects. It can also be used to automatically activate a Python virtualenv when entering projects too.
            • mise/JTBD/3 - Task Runner

              • mise Job/to be Done 3: as a task runner to share common tasks within a project
                • for example, run a task when a file changes
          • mise/Docs/Guides/Getting Started
            • mise/Todos
          • mise/Docs/Guides/Walkthrouh

            • mise commands

              • mise/install
    • hk
      • {{tweet https://x.com/jdxcode/status/1891606050119278853}}