paper

https://storage.googleapis.com/deepmind-media/Era-of-Experience%20/The%20Era%20of%20Experience%20Paper.pdf

This paper claims that the next big leap in AI won’t happen by giving models more human generated data. AI agents must learn through their own real world experiences instead of relying on human data. We already use RL (reinforcement learning). For example AlphaGo learned by playing millions of games, and coding agents can learn from whether tests pass. The idea isn’t new to reinforcement learning, but moving from AI that learns what humans know to AI that discovers what actually works. It also mentions that the big challenge is making that continuous experimentation safe in the real world.

Imagine an AI learning to grow cherries. Today, it might read thousands of gardening books and tell you how much water and sunlight cherries need. It will take the human knowledge of the physical world as the base fact. In the Era of Experience, the AI could grow cherries itself, try different conditions, observe the results, and discover better methods that nobody has written about. It’s a feedback mechanism involves real world experience.

Today’s RL often optimizes models using predefined tasks, reward models, preferences of human, or automatically verified answers. In the paper, for the proposed “era of experience”, AI learns continuously from the consequences of its own actions in an environment, potentially over months or years.

The authors defines the AI eras: 1. Era of Simulation: AI learns by trial and error in simulated environments. Examples: AlphaGo, AlphaZero, Atari. Very powerful, but mostly in narrowly defined problems with clear rewards. 2. Era of Human Data: AI learns from human-written text, code, demonstrations, and preferences. Examples: GPT-style language models. Broadly capable, but constrained by the information and assumptions available in human data. 3. Era of Experience (their prediction): AI interacts continuously with the real world, tests hypotheses, receives feedback, and learns strategies that humans may never have imagined.

A sketch chronology of dominant AI paradigms. The y-axis suggests the proportion of the field’s total effort and computation that is focused on RL.:

In the era of experience (direct quote from the paper)

• Agents will inhabit streams of experience, rather than short snippets of interaction.
• Their actions and observations will be richly grounded in the environment, rather than interacting via human dialogue alone.
• Their rewards will be grounded in their experience of the environment, rather than coming from human prejudgement.
• They will plan and/or reason about experience, rather than reasoning solely in human terms

To achieve this era the agents must interact with the physical world (direct quote):

To progress significantly further, a new source of data is required. This data must be generated in a way that continually improves as the agent becomes stronger; any static procedure for synthetically generating data will quickly become outstripped. This can be achieved by allowing agents to learn continually from their own experience, i.e., data that is generated by the agent interacting with its environment.

I saw this paper from a Youtube video by WelchLabs. There in the video, an interesting queston was asked as a thought experiment. Actually the experiment also coming from the paper itself.

This is my favorite LLM thought experiment. If you trained an LLM using a really old cutoff date for your training data, like 1000 AD, what would happen? If you asked your 1000 AD cutoff model what made metal shiny, it would probably reason in theistic terms (answer: The LORD made that way). If you went further back to some of humanity’s oldest texts, it would probably answer in terms of animism (answer: animal spirits inside metal). If your cutoff was around 1700, this LLM would likely explain metal shyness using Newtonian physics (answer: light reflecting on smooth surface), and a more modern model would answer in terms of quantum physics (answer: free electrons absorbing and re-emiting photons).

From CHI ‘08: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, April 5-10, 2008 · Florence, Italy

Paper https://dl.acm.org/doi/epdf/10.1145/1357054.1357072

We performed an empirical study to investigate whether thecontext of interruptions makes a difference. We found thatcontext does not make a difference but surprisingly, people completed interrupted tasks in less time with no differencein quality. Our data suggests that people compensate forinterruptions by working faster, but this comes at a price:experiencing more stress, higher frustration, time pressureand effort. Individual differences exist in the managementof interruptions: personality measures of openness toexperience and need for personal structure predict disruption costs of interruptions. We discuss implicationsfor how system design can support interrupted work

Also kind of a related tool: Interruptions impact on deep work simulator

Full paper - PDF

This article examines racist attitudes toward Atatu¨rk and Kemalism from the 1930s to the 1960s. Liberal, leftist and conservative-Islamist critics of republican Turkey’s founder and his policies have contributed to a widely shared image that, even if Kemalism was not essentially racist, the Kemalist approach to religious and ethnic minorities could hardly be described as egalitarian. Thus one is taken by surprise to uncover a parallel layer of virulent racist criticism, hidden under the deposit of decades of anti-Kemalist discourse

Full paper - PDF

Dialectical logic was founded by German famous philosopher F. Hegel, but it has not been laid on mathematics for a long time. In this paper author explains the dialectical logic pure mathematically, and shows that the classic formal logic, its mathematical expression is Boolean algebra(includes multiple value system), is a special case from dialectical logic, and the true-valued function for dialectical logic is a continuous function valued on closed interval and defined on time-space axes system. The Aristotle three laws of formal logic are expanded into expression of dialectical logic, and Russell paradox is expanded into the case of multiple order. Some new theorems for Boolean operators and the matrix expression for De Morgan’s theorem of multiple variables dialectical logic are given. At the end of the paper, linear or nonlinear dialectical logic are defined and analysis properties of dialectical logic true-valued function are pointed.

Heavy Metal in a Muslim Context - Pierre Hecker Full paper - PDF