minor docs clean up (#1269)

CONTRIBUTING.md

@@ -21,17 +21,20 @@ is being developed as a community effort thanks to all our volunteers, collabora
 listed here in alphabetical order:
 
 * Resmi Arjun
+* Omar Benjelloun (Google)
 * Alice Cheng (Nvidia)
 * Jiahao Chen (MIT)
 * Ramesh N Chukka (Intel)
 * Ray DeMoss (One Stop Systems)
 * Ryan T DeRue (Purdue University)
 * Himanshu Dutta (Indian Institute of Technology)
+* Nicolas Essayan
 * Justin Faust (One Stop Systems)
 * Diane Feddema (Red Hat)
 * Grigori Fursin (cTuning.org and cKnowledge.org)
 * Leonid Fursin (United Silicon Carbide)
 * Anirban Ghosh (Nvidia)
+* James Goel (Qualcomm)
 * Michael Goin (Neural Magic)
 * Jose Armando Hernandez (Paris Saclay University)
 * Mehrdad Hessar (OctoML)
@@ -40,6 +43,7 @@ listed here in alphabetical order:
 * Tom Jablin (Google)
 * Nino Jacob
 * David Kanter (MLCommons)
+* Alex Karargyris
 * Jason Knight (OctoML)
 * Ilya Kozulin (Deelvin)
 * @makaveli10 (Collabora)
@@ -63,15 +67,18 @@ listed here in alphabetical order:
 * Sergey Serebryakov (HPE)
 * Warren Schultz (Principled Technologies)
 * Amrutha Sheleenderan (Kerala Technical University)
+* Micah J Sheller (Intel)
 * Byoungjun Seo (TTA)
 * Aditya Kumar Shaw (Indian Institute of Science)
 * Ilya Slavutin (Deelvin)
 * Jinho Suh (Nvidia)
-* Arjun Suresh (cTuning.org and cKnowledge.org)
+* Arjun Suresh
 * Badhri Narayanan Suresh (Intel)
 * David Tafur (MLCommons)
 * Chloe Tessier
 * Gaurav Verma (Stony Brook University)
+* Zixian Wang
+* Nathan Wasson
 * Scott Wasson (MLCommons)
 * Haoyang Zhang  (UIUC)
 * Bojian Zheng (University of Toronto)

README.md

@@ -61,13 +61,20 @@ CK consists of several ongoing sub-projects:
 
   * [MLCommons docs to run MLPerf inference benchmarks from command line via CM](https://docs.mlcommons.org/inference)
 
+### Incubator
+
+We are preparing new projects based on user feedback:
+* [The next generation of CM](_incubator/cm-next-gen) *(prototyping stage)*
+* [The crowd-testing infrastructure for CM4MLOps and CM4MLPerf](_incubator/cm4mlops-testing) *(brainstorming stage)*
+
+
 ### License
 
 [Apache 2.0](LICENSE.md)
 
 ### Documentation
 
-**MLCommons is updating the CM documentation based on user feedback - please stay tuned for more details**.
+**MLCommons is updating the CM documentation based on user feedback - please check stay tuned for more details**.
 
 * [CM Getting Started Guide and FAQ](docs/getting-started.md)
   * [Common CM interface to run MLPerf inference benchmarks](docs/mlperf/inference)
@@ -79,21 +86,21 @@ CK consists of several ongoing sub-projects:
 * [CM and CK history](docs/history.md)
 
 
+### Citing CM
 
-### Acknowledgments
-
-Collective Knowledge (CK) and Collective Mind (CM) were created by [Grigori Fursin](https://cKnowledge.org/gfursin),
-sponsored by cKnowledge.org and cTuning.org, and donated to MLCommons to benefit everyone. 
-Since then, this open-source technology (CM, CM4MLOps, CM4ABTF, CM4Research, etc)
-is being developed as a community effort thanks to all our
-[volunteers, collaborators and contributors](https://github.com/mlcommons/ck/blob/master/CONTRIBUTING.md)! 
+If you found CM useful, please cite this article: 
+[ [ArXiv](https://arxiv.org/abs/2406.16791) ], [ [BibTex](https://github.com/mlcommons/ck/blob/master/citation.bib) ].
 
 You can learn more about the motivation behind these projects from the following articles and presentations:
 
 * "Enabling more efficient and cost-effective AI/ML systems with Collective Mind, virtualized MLOps, MLPerf, Collective Knowledge Playground and reproducible optimization tournaments": [ [ArXiv](https://arxiv.org/abs/2406.16791) ] 
 * ACM REP'23 keynote about the MLCommons CM automation framework: [ [slides](https://doi.org/10.5281/zenodo.8105339) ] 
 * ACM TechTalk'21 about automating research projects: [ [YouTube](https://www.youtube.com/watch?v=7zpeIVwICa4) ] [ [slides](https://learning.acm.org/binaries/content/assets/leaning-center/webinar-slides/2021/grigorifursin_techtalk_slides.pdf) ]
 
-### Citing this project
+### Acknowledgments
 
-Please use this [BibTex file](https://github.com/mlcommons/ck/blob/master/citation.bib).
+Collective Knowledge (CK) and Collective Mind (CM) were created by [Grigori Fursin](https://cKnowledge.org/gfursin),
+sponsored by cKnowledge.org and cTuning.org, and donated to MLCommons to benefit everyone. 
+Since then, this open-source technology (CM, CM4MLOps, CM4MLPerf, CM4ABTF, CM4Research, etc)
+is being developed as a community effort thanks to all our
+[volunteers, collaborators and contributors](https://github.com/mlcommons/ck/blob/master/CONTRIBUTING.md)! 

_incubator/README.md

@@ -0,0 +1 @@
+TBD

_incubator/cm-next-gen/README.md

@@ -0,0 +1 @@
+# Prototyping the next generation of CM

_incubator/cm4mlops-testing/README.md

@@ -0,0 +1 @@
+Prototyping infrastructure to crowd-test [CM4MLOps scripts](https://access.cknowledge.org/playground/?action=scripts)

_incubator/cm4mlperf/README.md

@@ -0,0 +1,4 @@
+# CM4MLPerf: CM automation for MLPerf benchmarks
+
+The idea is to provide a standard PYPI package to run MLPerf via CM.
+

cm/CHANGES.md

@@ -1,3 +1,6 @@
+## V2.3.4
+   - minor documentation update
+
 ## V2.3.3
    - minor documentation update for MLPerf inference v4.1
 

cm/README.md

@@ -49,7 +49,7 @@ using a few CM commands:
 
 pip install cmind -U
 
-cm pull repo cknowledge@cm4mlops --branch=mlperf-inference
+cm pull repo mlcommons@cm4mlops --branch=dev
 
 cm run script "run-mlperf-inference _r4.1 _accuracy-only _short" \
    --device=cpu \
@@ -139,7 +139,7 @@ See more examples of CM scripts and workflows to download Stable Diffusion, GPT-
 ```bash
 pip install cmind -U
 
-cm pull repo mlcommons@cm4mlops --branch=mlperf-inference
+cm pull repo mlcommons@cm4mlops --branch=dev
 
 cm show repo
 
@@ -238,20 +238,21 @@ and how to implement and share new automations in your public or private project
 
 [Apache 2.0](LICENSE.md)
 
-### Acknowledgments
+### Citing CM
 
-Collective Knowledge (CK) and Collective Mind (CM) were created by [Grigori Fursin](https://cKnowledge.org/gfursin),
-sponsored by cKnowledge.org and cTuning.org, and donated to MLCommons to benefit everyone. 
-Since then, this open-source technology (CM, CM4MLOps, CM4ABTF, CM4Research, etc)
-is being developed as a community effort thanks to all our
-[volunteers, collaborators and contributors](https://github.com/mlcommons/ck/blob/master/CONTRIBUTING.md)! 
+If you found CM useful, please cite this article: 
+[ [ArXiv](https://arxiv.org/abs/2406.16791) ], [ [BibTex](https://github.com/mlcommons/ck/blob/master/citation.bib) ].
 
 You can learn more about the motivation behind these projects from the following articles and presentations:
 
 * "Enabling more efficient and cost-effective AI/ML systems with Collective Mind, virtualized MLOps, MLPerf, Collective Knowledge Playground and reproducible optimization tournaments": [ [ArXiv](https://arxiv.org/abs/2406.16791) ] 
 * ACM REP'23 keynote about the MLCommons CM automation framework: [ [slides](https://doi.org/10.5281/zenodo.8105339) ] 
 * ACM TechTalk'21 about automating research projects: [ [YouTube](https://www.youtube.com/watch?v=7zpeIVwICa4) ] [ [slides](https://learning.acm.org/binaries/content/assets/leaning-center/webinar-slides/2021/grigorifursin_techtalk_slides.pdf) ]
 
-### Citing this project
+### Acknowledgments
 
-Please use this [BibTex file](https://github.com/mlcommons/ck/blob/master/citation.bib).
+Collective Knowledge (CK) and Collective Mind (CM) were created by [Grigori Fursin](https://cKnowledge.org/gfursin),
+sponsored by cKnowledge.org and cTuning.org, and donated to MLCommons to benefit everyone. 
+Since then, this open-source technology (CM, CM4MLOps, CM4MLPerf, CM4ABTF, CM4Research, etc)
+is being developed as a community effort thanks to all our
+[volunteers, collaborators and contributors](https://github.com/mlcommons/ck/blob/master/CONTRIBUTING.md)! 

cm/cmind/__init__.py

@@ -2,7 +2,7 @@
 #
 # Written by Grigori Fursin
 
-__version__ = "2.3.3"
+__version__ = "2.3.4"
 
 from cmind.core import access
 from cmind.core import error

cm/cmind/repo/automation/repo/module.py

@@ -110,14 +110,14 @@ def pull(self, i):
             # unless branch/checkout is used - in such case we keep old repository
             # for backwards compatibility and reproducibility
 
-            r = net.request({'get': {'action': 'check-migration-repo-notes', 'repo': url}})
+            branch = i.get('branch', '')
+            checkout = i.get('checkout', '')
+
+            r = net.request({'get': {'action': 'check-migration-repo-notes', 'repo': url, 'branch': branch, 'checkout': checkout}})
             notes = r.get('dict', {}).get('notes','')
             if notes !='':
                 print (notes)
 
-            branch = i.get('branch', '')
-            checkout = i.get('checkout', '')
-
             if alias == 'mlcommons@ck' and branch == '' and checkout == '':
                 print ('=========================================================================')
                 print ('Warning: mlcommons@ck was automatically changed to mlcommons@cm4mlops.')

cm/setup.py

@@ -69,7 +69,7 @@ def run(self):
     name="cmind",
 
     author="Grigori Fursin",
-    author_email="Grigori.Fursin@cTuning.org",
+    author_email="gfursin@cKnowledge.org",
 
     version=version,
 
@@ -80,7 +80,7 @@ def run(self):
     long_description=open('README.md', encoding="utf-8").read(),
     long_description_content_type="text/markdown",
 
-    url="https://github.com/mlcommons/ck",
+    url="https://github.com/mlcommons/ck/tree/master/cm",
 
     python_requires="", # do not force for testing
 
@@ -107,5 +107,5 @@ def run(self):
 
     zip_safe=False,
 
-    keywords="collective mind,cmind,ck3,cdatabase,cmeta,automation,portability,reusability,productivity,meta,JSON,YAML,python,api,cli"
+    keywords="cm,collective mind,cmind,ck3,cdatabase,cmeta,automation,portability,reusability,productivity,meta,JSON,YAML,python,api,cli"
 )

docs/installation.md

@@ -26,7 +26,7 @@
 
 
 
-MLCommons Collective Mind framework requires minimal dependencies to run on any platform: `python 3+, pip, git, wget`.
+MLCommons Collective Mind framework requires minimal dependencies to run on any platform: `python 3+, pip, git, git-lfs, wget`.
 However, most CM automation recipes shared by the community and MLCommons require Python 3.7+ .
 
 ***By default, CM will pull Git repositories and cache installations and downloaded files in your `$HOME/CM` directory (Linux/MacOS). 
@@ -54,7 +54,7 @@ You can reuse misc CM utils listed [here](#misc-cm-utils).
 ```bash
 sudo apt update && sudo apt upgrade
 
-sudo apt install python3 python3-pip python3-venv git wget curl
+sudo apt install python3 python3-pip python3-venv git git-lfs wget curl
 sudo apt install libgl1-mesa-dev
 ```
 
@@ -95,7 +95,6 @@ Path to CM repositories:      D:\Work1\CM
 GitHub for CM developments:        https://github.com/mlcommons/ck/tree/master/cm
 GitHub for CM automation scripts:  https://github.com/mlcommons/cm4mlops
 Reporting issues and ideas:        https://github.com/mlcommons/ck/issues
-MLCommons taskforce developing CM: https://github.com/mlcommons/ck/blob/master/docs/taskforce.md
 
 ```
 
@@ -111,7 +110,7 @@ You are ready to use CM automation meta-framework.
 ```bash
 sudo dnf update
 
-sudo dnf install python3 python-pip git wget curl
+sudo dnf install python3 python-pip git git-lfs wget curl
 
 python3 -m pip install cmind --user
 
@@ -132,7 +131,7 @@ Don't forget to add brew to PATH environment as described in the end.
 Then install python, pip, git and wget:
 
 ```bash
-brew install python3 git wget curl
+brew install python3 git git-lfs wget curl
 
 python3 -m pip install cmind
 ```

docs/taskforce.md

@@ -2,8 +2,11 @@
 
 # MLCommons Task Force on Automation and Reproducibility
 
-**News (May 2024):** our task force has successfully accomplished its mission and our on-going developments 
- will be funded by MLCommons and integrated with several MLCommons Working Groups - please stay tuned for more details!
+**News (May 2024):** our task force has successfully accomplished the first goal 
+ to provide a [stable CM interface for MLPerf benchmarks](https://docs.mlcommons.org/inference) 
+ and discussing the next steps with MLCommons - please stay tuned for more details!
+
+
 
 ## Mission
 
@@ -22,6 +25,12 @@ We thank [cKnowledge.org](https://cKnowledge.org), [cTuning.org](https://cTuning
 and [MLCommons](https://mlcommons.org) for sponsoring this project!
 
 
+### Citing CM
+
+If you found CM useful, please cite this article: 
+[ [ArXiv](https://arxiv.org/abs/2406.16791) ], [ [BibTex](https://github.com/mlcommons/ck/blob/master/citation.bib) ].
+
+
 ## Current projects
 
 * Continue improving CM to support different MLCommons projects for universal benchmarking and optimization across different platforms.
@@ -111,3 +120,4 @@ after he donated his CK and CM automation technology to MLCommons in 2022 to ben
 Since then, this open-source technology is being developed as a community effort based on user feedback.
 We would like to thank all our [volunteers, collaborators and contributors](../CONTRIBUTING.md) 
 for their support, fruitful discussions, and useful feedback! 
+