Symbolic Behaviour Benchmark (S2B)

Last updated on Jul 16, 2022

A Meta-Referential Game.

Suite of OpenAI Gym-compatible multi-agent reinforcement learning environment centered around referntial games to benchmark for behavioral traits pertaining to symbolic behaviours, as described in Santoro et al., 2021, “Symbolic Behaviours in Artificial Intelligence”, primarily: exhibiting receptive, constructive, malleable, and separable behaviours.

Usage

gym must be installed. Environments can be created as follows, for instance, in order to test for receptivity and constructivity:

>>> import gym
>>> import symbolic_behaviour_benchmark
>>> env = gym.make(
        "SymbolicBehaviourBenchmark-ReceptiveConstructiveTestEnv-v0", 
        vocab_size=10,
        max_sentence_length=5,
        nbr_latents=5,
        min_nbr_values_per_latent=3,
        max_nbr_values_per_latent=5,
        nbr_object_centric_samples=1,
        nbr_distractors=3,
        use_communication_channel_permutations=True,
        allow_listener_query=False,
    )

Installation

Installing via pip

This package is available in PyPi as symbolic_behaviour_benchmark

pip install symbolic_behaviour_benchmark

Installing via cloning this repository

git clone https://www.github.com/Near32/SymbolicBehaviourBenchmark
cd SymbolicBehaviourBenchmark
pip install -e .

Kevin Denamganaï

Independent Researcher

My research investigates the conditions under which AI systems acquire and deploy structured symbolic representations — towards in-context grounding of novel atomic symbols and their systematic recombination into unseen configurations — spanning Compositional Generalisation, Formal Mathematics, Differentiable Language Models, and Physical Simulation. I have also investigated Language Emergence & Grounding (Emergent Communication), Unsupervised Representation Learning, Natural Language Processing, and Multi-Agent Deep Reinforcement Learning.