General
- Interesting essay: By default, capital will matter more than ever after AGI.
- Google DeepMind preprint: A theory of appropriateness with applications to generative artificial intelligence.
- Can one objectively define “good taste” (e.g. in appreciating art?). If one can (e.g. to objectively understand the details and context that explain human preferences), then it seems likely that AIs will eventually exhibit superhuman taste, in that they will be able to analyze given data from a multitude of well-informed perspectives.
Research Insights
- An interesting effect: fine-tuning GPT-4o on responses where the first letter of each line spells out H-E-L-L-O leads to a model that can correctly explain this underlying rule (even though the rule was never provided to it). This is surprising since when generating a reply, a token-wise prediction cannot “see ahead” and know that it will spell out HELLO; yet the LLM is somehow able to predict its own behavior, suggesting it has some knowledge of its own internal state.
- Further testing with the pattern HELOL gave far worse results, implying strong reliance on the existence of the HELLO pattern in the training data.
- Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs. The authors analyze whether we are efficiently using inference-time compute, and propose mitigate strategies to avoid overthinking.
AI Agents
- Huggingface introduce smolagents, a lightweight framework for agents.
- Agentarium is a Python framework for orchestrating agents.
- Eliza is a framework for AI models to access resources (documents, Discord, Twitter, etc.).
Audio
- TangoFlux: Super Fast and Faithful Text to Audio Generation with Flow Matching and Clap-Ranked Preference Optimization (code).
3D
- zoo.dev is developing workflows for CAD where one can switch between generative and traditional-edit modes.
Science
- Where did viruses come from? AlphaFold and other AIs are finding answers.
- There is growing sophistication in bio-polymer prediction methods: AlphaFold, AlphaFold 2, AlphaFold 3, ESMFold, Evo, Chroma.
Robots
- LimX released a video of a new humanoid design.
- EngineAI released details of their PM01 design (c.f. existing SE01 design sells for $12,000).