一款基于强化学习的外交策略游戏。结合了类似于 AlphaGo 的战略推理能力，和类似于 GPT-3 的语言组织能力。在每场比赛中，它都会查看比赛状态，各位玩家的对话历史，从而预测其他玩家的。它能自行制定计划，并通过人类语言与其他玩家进行协调执行自己的策略。
|Colin Flaherty 7a9acebf80 should be .txt not .md||3 months ago|
|.circleci||4 months ago|
|bin||4 months ago|
|conf||4 months ago|
|data||3 months ago|
|dipcc||4 months ago|
|docs||4 months ago|
|fairdiplomacy||4 months ago|
|fairdiplomacy_external||4 months ago|
|heyhi||4 months ago|
|model_cards||4 months ago|
|parlai_diplomacy||4 months ago|
|slurm||4 months ago|
|thirdparty||4 months ago|
|unit_tests||4 months ago|
|.flake8||4 months ago|
|.gitattributes||4 months ago|
|.gitignore||4 months ago|
|.gitmodules||4 months ago|
|.pre-commit-config.yaml||4 months ago|
|CODE_OF_CONDUCT.md||4 months ago|
|CONTRIBUTING.md||4 months ago|
|LICENSE.md||4 months ago|
|LICENSE_FOR_MODEL_WEIGHTS.txt||3 months ago|
|Makefile||4 months ago|
|README.md||3 months ago|
|pyproject.toml||4 months ago|
|pyrightconfig.CI.json||4 months ago|
|pyrightconfig.noexcludes.json||4 months ago|
|requirements.txt||4 months ago|
|run.py||4 months ago|
|setup.py||4 months ago|
This code contains checkpoints and training code the following papers:
A very brief orientation:
Diplomacy is a strategic board game set in 1914 Europe. The board is divided into fifty-six land regions and nineteen sea regions. Forty-two of the land regions are divided among the seven Great Powers of the game: Austria-Hungary, England, France, Germany, Italy, Russia, and Turkey. The remaining fourteen land regions are neutral at the start of the game.
Each power controls some regions and some units. The number of the units controlled depends on the number of the controlled key regions called Supply Centers (SCs). Simply put, more SCs means more units. The goal of the game is to control more than half of all SCs by moving units into these regions and convincing other players to support you.
You can find the full rules here. To get the game's spirit, watch some games with comments. You can play the game online on webDiplomacy either against bots or humans.
# Clone the repo with submodules: git clone --recursive firstname.lastname@example.org:facebookresearch/diplomacy_cicero.git diplomacy_cicero cd diplomacy_cicero # Apt installs apt-get install -y wget bzip2 ca-certificates curl git build-essential clang-format-8 git wget cmake build-essential autoconf libtool pkg-config libgoogle-glog-dev # Install conda wget --quiet https://repo.anaconda.com/miniconda/Miniconda3-4.7.10-Linux-x86_64.sh -O ~/miniconda.sh /bin/bash ~/miniconda.sh -b # Create conda env conda create --yes -n diplomacy_cicero python=3.7 conda activate diplomacy_cicero # Install pytorch, pybind11 conda install --yes pytorch=1.7.1 torchvision cudatoolkit=11.0 -c pytorch conda install --yes pybind11 # Install go for boringssl in grpc # We have some hacky patching code for protobuf that is not guaranteed # to work on versions other than this. conda install --yes go protobuf=3.19.1 # Install python requirements pip install -r requirements.txt # Local pip installs pip install -e ./thirdparty/github/fairinternal/postman/nest/ # NOTE: Postman here links against pytorch for tensors, for this to work you may # need to separately have installed cuda 11 on your own. pip install -e ./thirdparty/github/fairinternal/postman/postman/ pip install -e . -vv # Make make # Run unit tests make test_fast
After each pull it's recommended to run
make to re-compile internal C++ and protobuf code.
Please email email@example.com to request the password. Then run
bash bin/download_model_files.sh <PASSWORD>. This will download and decrypt all relevant model files into
./models. This might take awhile. Please note the model files have their own license separate from the code in this repository. More details on this can be found below.
JSON data and visualizations for games that Cicero played in are located in data/cicero_redacted_games. Only conversations with players who have consented to having their dialogue released are included. Please refer to the (separately-licensed) fairdiplomacy_external subdirectory for details on HTML visualizations.
The front-end for most tasks is
run.py, which can run various tasks specified by a protobuf config. The config schema can be found at
conf/conf.proto, and example configs for different tasks can be found in the
conf folder. This can be used for most tasks (except training parlai models): training no-press models, comparing agents, profiling things, launching an agent on webdip, etc.
The config specification framework, called HeyHi, is explained here
A core abstraction is an
Agent, which is specified by an
Agent config whose schema lives in
To simulate 1v6 games between a pair of agents, you can run the
compare_agents task. For example, to play one Cicero agent as Turkey against six full-press imitation agents, you can run
python run.py --adhoc --cfg conf/c01_ag_cmp/cmp.prototxt Iagent_one=agents/bqre1p_parlai_20220819_cicero_2.prototxt Iagent_six=agents/ablations/cicero_imitation_only.prototxt power_one=TURKEY
If you don't have sufficient memory to load two agents, you can load a single agent in self-play with the
python run.py --adhoc --cfg conf/c01_ag_cmp/cmp.prototxt Iagent_one=agents/bqre1p_parlai_20220819_cicero_2.prototxt use_shared_agent=1 power_one=TURKEY
To run the training for Cicero and/or Diplodocus:
python run.py —adhoc —cfg conf/c04_exploit/research_20221001_paper_cicero.prototxt launcher.slurm.num_gpus=256 python run.py —adhoc —cfg conf/c04_exploit/research_20221001_paper_diplodocus_high.prototxt launcher.slurm.num_gpus=256
The above training commands are designed for running on an appropriately configured Slurm cluster with a fast cross-machine shared filesystem. One can also instead pass
launcher.local.use_local=true to run them on locally, e.g. on an individual 8-GPU-or-more GPU machine but training may be very slow.
See here for some separately-licensed code for rendering game jsons with HTML, as well as connecting agents to run on webdiplomacy.net.
Supervised training and/or behavioral cloning for various dialogue-conditional models as well as pre-RL baseline dialogue-free models involves some of the scripts in parlai_diplomacy via the ParlAI framework, and on the dialogue-free side, some of the configs conf/c02_sup_train and train_sl.py. However the dataset of human games and/or dialogue is NOT available here, so the relevant code and configs are likely to be of limited use. They are provided here mostly as documentation for posterity.
However, as mentioned above pre-trained models are available, and with sufficient compute power, re-running the RL on top of these pre-trained models is also possible without any exteral game data.
pre-commit install to install pre-commit hooks that will auto-format python code before commiting it.
Or you can do this manually. Use black auto-formatter to format all python code.
For protobufs use
clang-format-8 conf/*.proto -i.
To run tests locally run
We have 2 level of tests: fast, unit tests (run with
make test_fast) and slow, integration tests (run with
The latter aims to use the same entry point as users do, i.e.,
run.py for the HeyHi part and
diplom for the ParlAi.
pytest to run and discover tests. Some useful pytest commands.
To run all tests in your current directory, simply run:
To run tests from a specific file, run:
To use name-based filtering to run tests, use the flag
-k. For example, to only run tests with
parlai in the name, run:
pytest -k parlai
For verbose testing logs, use
pytest -v -k parlai
To print the output from a test or set of tests, use
-s; this also allows you to set breakpoints:
To view the durations of all tests, run with the flag
pytest --durations=0 unit_tests/
The following license, which is also available here, covers the content in this repo except for the fairdiplomacy_external directory. The content of fairdiplomacy_external is separately licenced under a version of the AGPL, see the license file within that directory for details.
(covers this repo except for the fairdiplomacy_external directory) MIT License Copyright (c) Meta, Inc. and its affiliates. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
We are releasing model weights under a separate license: CC-BY-NC (version 4.0). This license is copied into this repository for convenience: LICENSE_FOR_MODEL_WEIGHTS.txt.