diff --git a/doc/src/SUMMARY.md b/doc/src/SUMMARY.md index c3a67212..aed3d3f5 100644 --- a/doc/src/SUMMARY.md +++ b/doc/src/SUMMARY.md @@ -187,7 +187,6 @@ - [What's Next](learning_material/dot_ix_diagram_generator/whats_next.md) - [Q & A](learning_material/dot_ix_diagram_generator/qna.md) - [Peace: Zero Stress Automation](learning_material/peace_zero_stress_automation.md) - - [🕊️ Peace Framework](learning_material/peace_zero_stress_automation/peace_framework.md) - [🐉 Origin Story](learning_material/peace_zero_stress_automation/origin_story.md) - [Manual Deployment](learning_material/peace_zero_stress_automation/origin_story/manual_deployment.md) - [Automation](learning_material/peace_zero_stress_automation/origin_story/automation.md) @@ -195,6 +194,7 @@ - [Engineering Success](learning_material/peace_zero_stress_automation/origin_story/engineering_success.md) - [Automation (For Real)](learning_material/peace_zero_stress_automation/origin_story/automation_for_real.md) - [Human Eyes](learning_material/peace_zero_stress_automation/origin_story/human_eyes.md) + - [🕊️ Peace Framework](learning_material/peace_zero_stress_automation/peace_framework.md) - [🏗️ envman (example tool)](learning_material/peace_zero_stress_automation/envman_example_tool.md) - [💡 Understandability](learning_material/peace_zero_stress_automation/understandability.md) - [Demo](learning_material/peace_zero_stress_automation/understandability/demo.md) @@ -213,15 +213,15 @@ - [🏭 Architecture](learning_material/peace_zero_stress_automation/architecture.md) - [Components](learning_material/peace_zero_stress_automation/architecture/components.md) - [Item](learning_material/peace_zero_stress_automation/architecture/item.md) - - [Item Implementations](learning_material/peace_zero_stress_automation/architecture/item_implementations.md) + - [Flow](learning_material/peace_zero_stress_automation/architecture/flow.md) - [Commands](learning_material/peace_zero_stress_automation/architecture/commands.md) - - [Item Parameters](learning_material/peace_zero_stress_automation/architecture/item_parameters.md) + - [Tool](learning_material/peace_zero_stress_automation/architecture/tool.md) - - - [🪆 Side-Side Projects](learning_material/peace_zero_stress_automation/side_side_projects.md) - - [🛑 Interruptible](learning_material/peace_zero_stress_automation/side_side_projects/interruptible.md) - - [🎨 Dot IX](learning_material/peace_zero_stress_automation/side_side_projects/dot_ix.md) + + + + - [🦖 Project Status](learning_material/peace_zero_stress_automation/project_status.md) - [Readiness](learning_material/peace_zero_stress_automation/project_status/readiness.md) - [🔗 Links](learning_material/peace_zero_stress_automation/links.md) diff --git a/doc/src/learning_material/peace_zero_stress_automation.md b/doc/src/learning_material/peace_zero_stress_automation.md index fa413652..4f16e9fe 100644 --- a/doc/src/learning_material/peace_zero_stress_automation.md +++ b/doc/src/learning_material/peace_zero_stress_automation.md @@ -19,7 +19,7 @@
https://peace.mk/
Azriel Hoh
-
October 2024
+
February 2025
@@ -55,77 +55,71 @@ cd ([$env.TEMP demo envman] | path join) 1. Heya everyone, my name is Azriel, and today I'll be showing you my automation side project, called Peace. -2. Peace is a framework to create user friendly automation. -3. It provides a set of constraints and common functionality, that when you write code according to these constraints, and plug them into the common functionality, you get a nice tool. -4. As with every side project, there is an origin story. - -5. In my first job, a team of us were given the task of fully automating our solution deployment. -6. So this was the deployment process, it's all manual. - -7. and, obviously this is the solution. Genius. - -8. We wanted this process: click, wait, done. -9. And through engineering eyes, we aimed for:. *(linking: Dimensions)* -10. End-to-end automation. -11. Repeatable correctness. -12. and Performance. -13. and we delivered! - -14. If you measure success using these metrics, it was undeniable. -15. We reduced the deployment duration down from 2 weeks of manual steps, to 30 minutes. -16. However, our users said "we hate this!". Azriel, this doesn't enhance our lives. - -17. What we really did when we introduced automation, was this. -18. When switching from manual steps to automation, the work changes from doing each step at your own pace, to setting up the parameters for all of the steps, pressing go, and waiting. -19. When it's done, you check if your parameters were correct. -20. If they're correct, that's fine. -21. If they weren't, then you had to understand the error, figure out which step it came from, and which parameters feed into that step. -22. Then when the parameter is fixed, which may take 30 seconds, they still had to wait 30 minutes to confirm if their fix worked, -23. and this delayed feedback loop was frustrating. -24. For new users, it was especially painful: -25. We're telling them to fill in parameters that they don't understand, -26. to feed into a process that they cannot see, -27. to create an environment, that they cannot visualize. -28. So they may not have understood what they were doing, but it was certainly our fault. -29. We created pain. - -30. We had engineering eyes, but not human eyes: -31. We took away understandability, -32. We took away control. -33. And when you take away understanding and control, you inadvertently also take away morale. *What little they had.* - -34. Ideally we should have built something that provides the benefits of automation, -35. while retaining the benefits of manual execution. -36. This is what the Peace framework aims to do. -37. And today I'd like to show you how it does this, through a tool built using the Peace framework. - -38. It's called `envman`, short for environment manager. -39. `envman` automates the download of a web application from github, creates some resources in Amazon, and uploads the web application. -40. Notably there's a missing step to launch a server that runs the web application, but I'm all out of AWS credits. So if you have spare, I'd gladly take them. - -41. The first thing we took away was understandability, so let's put that back. - -42. There are two ways we tend to write automation: -43. Either we produce too little information, and we can't tell what's going on, -44. or, we produce too much information, and we still can't tell what's going on. -45. For understandability, we need to have something in between. - -46. Let's take a look. -47. This is what it looks like when you have too little information: -48. `clear`, `./envman deploy --format none`, clean. -49. Something's going on, I promise! -50. This is what it looks like when you have too much information: -51. `./envman deploy --format json`, clean. -52. And finally, something in between. -53. See if you can see how many steps there are in this process, and whether they complete successfully: -54. `clear`, `./envman deploy`. -55. How many steps are there? *gesture* -56. Did every step complete successfully? -57. "Green means good", so I believe so. -58. What resources were created? *gesture* -59. And if we clean up the environment, you'll see a similar interface, so you can tell that each resource is deleted: `./envman clean`. -60. That's all good when things go well, but what happens in a failure? Can we understand it? -61. First we'll limit the connection speed of the tool to 40 kilobits per second: +2. Peace is a framework to create zero stress automation. +3. As with every side project, there is an origin story. + +4. In my first job, a team of us were given the task of fully automating our solution deployment. +5. So this was the deployment process, it's all manual. + +6. and, obviously this is the solution. Genius. + +7. We wanted this process: click, wait, done. +8. Being good engineers, we aimed for:. *(linking: Dimensions)* +9. End-to-end automation. +10. Repeatable correctness. +11. and Performance. +12. and we delivered! + +13. If you measure success using these metrics, it was undeniable. +14. We reduced the deployment duration down from 2 weeks, to 30 minutes. +15. However, our users said "we hate this!". Azriel, this doesn't enhance our lives. + +16. What we really did when we introduced automation, was this. +17. When switching from manual steps to automation, the work changes from doing each step at your own pace, to setting up the parameters for all of the steps, pressing go, and waiting. +18. If it didn't work, then you had to clean up and start again. +19. And for people who were unfamiliar with the process, it was especially painful: +20. We're telling them to fill in parameters that they don't understand, +21. to feed into a process that they cannot see, +22. to create an environment, that they cannot visualize. +23. So they may not have understood what they were doing, but it was certainly our fault. +24. We created pain. + +25. We had engineering eyes, but not human eyes: +26. We took away understandability and control. +27. And when you take those away, you inadvertently also take away morale. *What little they had.* + +28. Ideally we should have built something that provides the value of automation, +29. while retaining the value of manual execution. +30. This is what the Peace framework aims to do. + +31. Normally when we write automation, we spend just enough effort to get things working. +32. I wanted a framework which, by spending that same amount of effort, I would get a much nicer tool. + +33. So today I'd like to show you `envman`, a tool built using the Peace framework. +34. `envman` downloads a web application from github, creates some resources in Amazon, and uploads the web application. +35. Notably there's a missing step to launch a server that runs the web application, but that's not free. + +36. The first thing we took away was understandability, so let's put that back. + +37. There are two ways we tend to write automation: +38. Either we produce too little information, and we can't tell what's going on, +39. or, we produce too much information, and we still can't tell what's going on. +40. For understandability, we need to have something in between. + +41. This is what it looks like when you have too little information: +42. `clear`, `./envman deploy --format none`, clean. +43. This is what it looks like when you have too much information: +44. `./envman clean --format json`, clean. +45. The right balance is somewhere in between. +46. `clear`, `./envman deploy`. +47. How many steps are there in this process? *gesture* +48. Did it work? +49. "Green means good", so of course! +50. What resources were created? *gesture* +51. And if we clean up the environment, you'll see a similar interface, so you can tell that each resource is deleted: `./envman clean`. + +52. That's all good when things go well, but what happens in a failure? Can we understand it? +53. First we'll limit the connection speed of the tool to 40 kilobytes per second: ```sh New-NetQosPolicy ` @@ -135,16 +129,16 @@ cd ([$env.TEMP demo envman] | path join) -ThrottleRateActionBitsPerSecond 40KB ``` -62. and run the deployment again: `clear`, `./envman deploy`. -63. You can see that our download from github has slowed, -64. and in a little while we should see an error happen. -65. Here we go. -66. With fresh eyes, can you see which step went wrong? -67. Red means bad, so it should be apparent. -68. In detail, what went wrong, why it went wrong, and how to recover, are all shown. -69. We failed to upload the object. Why? The upload timed out, and make sure you are connected to the internet and try again. -70. We're also shown which resources exist and which don't, so we don't have to guess. -71. If we fix our connection, and re-run the automation: +54. and run the deployment again: `clear`, `./envman deploy`. +55. You can see that our download from github has slowed, +56. and in a little while we should see an error happen. +57. Here we go. +58. Can you see which step went wrong? +59. Red means bad, so that one. +60. In detail, what went wrong, why it went wrong, and how to recover, are all shown. +61. We failed to upload the object. Why? The upload timed out, and make sure you are connected to the internet and try again. +62. We're also shown which resources exist and which don't, so we don't have to guess. +63. If we fix our connection, and re-run the automation: ```sh Remove-NetQosPolicy ` @@ -153,250 +147,172 @@ cd ([$env.TEMP demo envman] | path join) -Confirm:$false ``` -72. You'll see that it picks up where it left off, and completes the process. -73. That is, what you think it should do, it does. No surprises. - -74. So in summary, with information, the goldilocks principle applies: -75. Too much information is overwhelming, too little is not useful, and there's some middle ground which is just right. -76. The Peace framework generally tries to fit the most relevant information on one screen. - -77. The second thing we took away, was control. - -78. Most automation tools give you one button -- start -- and that's it. -79. Start the creation, or update, and start the deletion. -80. While pressing start is not difficult, knowing whether the automation will do what we think it will, before we press start, is difficult. - -81. What we should understand *before* starting anything, is: -82. Where we are -- our current state, -83. Where we want to go -- our goal state, and -84. The distance between the two. -85. Because if we start with nothing, and end up with something, the distance is something. -86. And if we start with something, and our goal state is something, the distance is nothing. -87. And if we start with something, and our goal state is something else, the distance is that *else* -88. When we understand these three things, then we can make an informed decision if we should press go. -89. Now, if we press start, and change our mind, can we stop the process? -90. Without automation, we can. -91. Like, if someone said, "Azriel! Stop work." -92. I'd say, "Gladly." I can stop where I am. - -93. With automation, you need to intentionally build interruptibility into the process. -94. And while pressing Ctrl C on a command line tool is one form of interruption, -95. what we really care about, is safe interruption. -96. i.e. Stop what you're doing when it is safe to do so. -97. Maybe we're at step 5 of a 10 step process, and we want to adjust the parameter for step 7. -98. If we can interrupt the process, adjust the parameters, press go, and have the automation pick up where it left off, that would be great. -99. As in, don't undo all of the work you've already done to get to this point. -100. I just want to fix the parameter for the later step, and continue. - -101. Let's see all of this control, in action. -102. Before we run our deployment, what is our environment's current state. -103. Just like we can run `git status`, we can also run `./envman status`. -104. What state will the automation bring our environment to, when we run it? `./envman goal` -105. What's the difference? `./envman diff` -106. The commands are intentionally similar to `git` commands so we make use of familiar names. -107. And for interruptibility, when we deploy, we'll stop the process halfway. -108. `./envman deploy`, ctrl c. - -109. Here you can see steps 1 through 3, and step 5 were complete, -110. and step 4 and 6 were not started due to the interruption. -111. If we look at the diff: `./envman diff`, -112. you can see that steps 1, 2, 3, and 5 are done, steps 4 and 6 haven't been executed. -113. If we change our parameters, to using version 0.1.2 instead of 0.1.1 of our web application, -114. the diff will now show that step 1 will change. -115. And if we run deploy again, that is exactly what happens. - -116. When cleaning up, we can also interrupt the process. -117. Steps 1, 4, 5, and 6 were cleaned, and 2 and 3 were not. -118. And we can choose to either deploy the environment again, or clean up fully. -119. Let's deploy it to completion. `deploy`, `clean`. - -120. What's the use of this? -121. Well there was once we were told, -122. "hey this customer doesn't need their environment anymore, you can delete it." -123. "You sure?" -124. "Yes." -125. So we started the deletion process, and we got this "Hey stop. Stop what you're doing." -126. "We can't. It's all just going to go." -127. And that was the beginning of a very exciting day. -128. So, build a stop button into your automation people. -129. If you use Peace, it is built in for you. - -130. We've given back to the user some control, but there are other things still to be implemented like running a subset of the process. -131. Not too hard to implement, just needs time. - -132. Morale. -133. Not everyone who uses automation tools has a software background, and not everyone uses the command line all the time. -134. So why not create something that caters for these situations as well? - -135. Back to understandability, normally when explaining what automation does, -136. we tend to draw a diagram on the whiteboard, -137. or create a diagram in an internal documentation site. -138. However, it's never really accurate, and it's usually a tangle of overlapping boxes and lines, -139. so it is hard to understand, because the information isn't clear. - -140. So here's a web interface. `./envman web` -141. Based on the code written for your automation, two diagrams are generated: -142. The one on the left is called the Progress diagram, which shows the steps in your process, -143. and the one on the right is the Outcome diagram, which shows what the deployed environment looks like, before you deploy it. -144. By clicking on these steps on the right, we get to see what is happening in that step. - -145. For example the first step is to download a file from Github, it shows you the request to github and where it saves the file on the file system. -146. Then it creates the IAM policy, role, and instance profile, and S3 bucket, -147. then uploads the web application to that bucket. -148. All of this is generated from your automation code. Magic. -149. This is what you can use to teach someone, or self learn, what the automation process is, and what the environment looks like. -150. And you don't have to keep erasing and redrawing lines on the whiteboard. -151. Which step was unclear? This one? Let's go through that again. - -152. Now, this is great, but I like this one. -153. The diagram you saw is the example environment, but what does the actual environment look like? -154. We can discover it. -155. The diagram on the right has faded boxes for each resource, indicating that it doesn't exist. -156. When I click deploy, you can watch the progress diagram on the left, which will show you which steps are being executed, -157. or you can watch the outcome diagram on the right, which will show you the interactions between hosts, that are happening in real time. - -158. All of the steps completed successfully, that why they're green, -159. and the resources have been created, so they are now visible. -160. We can do the same for clean up, and it will delete all of the resources from Amazon, as well as on disk. - -161. And if we were to have an error, as we did before, we should see it clearly. -162. *slow down internet, click deploy* -163. Let's take a moment to admire this diagram. -164. Ooh look it's gone red. -165. So very quickly, from the user interface, you can tell which step the error came from, -166. as well as which resources it involves. -167. And we can surface the timeout message on the web interface, I just haven't coded that part yet. -168. Cool. - -169. So for morale, a lot of effort has been put into aesthetics. -170. For seeing the state of the system, showing one line for each resource, with a link to the full detail, is deliberate. -171. If you've ever been on-call and gotten a call out in the middle of the night, it's very annoying to have to go and find each resource that is part of the system you are investigating. -172. If I can think it, take me there. - -173. For progress, we present the information at a level of detail that is digestable, - -174. and for errors, instead of panicking, which is visually equivalent of printing a stack trace, -175. we take that error, refine it, and make it beautiful. -176. Always include what went wrong, the reason, and how to recover, -177. because when help people recover from a bad situation, -178. you recover their morale. - -179. With all of these aesthetic refinements, that box, is no longer opaque. -180. It is completely, clear. -181. You can see inside it, you can understand it, and you can control it. - -182. How does all of this work? -183. Magic. - -184. Architecture, how does it fit together? - -185. The Peace framework is categorised into two main parts. -186. The item definition, which is the common shape of logic and data, for anything that is managed by automation, and -187. Common functionality, which works with those shapes to provide command execution and a user interface. -188. Item crates contain the logic and data to automate one thing, and -189. the tool crate connects different items together, and passes them to the common functionality from the Peace framework, to provide automation. -190. These groupings are deliberate, so that you can share and reuse common item logic from the standard package registry, -191. while keeping proprietary values and workflows within your tool. -192. Let's go deeper. - -193. Starting with Item. -194. If you think of one step in a process, normally we would write code to *do* the step. -195. But instead of only writing code that does the work of that step, -196. an Item is a collection of functions that interact with the thing that is being automated. -197. What is the current state of the thing I'm managing? -198. What will it be, after the automation logic is executed? -199. What's the difference between these states? -200. What does it look like if it's not there? -201. The actual work logic, and -202. interactions -- what are the hosts, and paths that are involved in this automation. -203. Is it a request to fetch data back in, or is it a push to push data out. -204. This information is used to generate the diagram you saw earlier. - -205. An example implementation of this, the File Download. -206. The current state function returns the state of the file on disk -- whether or not it exists. -207. And if it does exist, it also returns the MD5 hash. -208. The goal state function returns the state of the file from the server, because the state of the file on the server, will become the state of the file on disk, when the download is executed. -209. So this would fetch the content-length and etag from the server, as a way to compare with what is on disk locally. -210. Many servers use the MD5 hash of a file as its etag. -211. `state_diff` returns whether the local file has the same hash as the remote file. -212. If it's got a different hash, then we assume we need to download it. -213. `state_clean` returns "the file does not exist". -214. `apply` downloads the file. -215. and `interactions` says I'm pulling data from this host, and writing to this path on localhost. - -216. A collection of functions is called an Item. -217. And a collection of items, is called a Flow. -218. And a flow also contains the dependency ordering between items. -219. And in Rust, since we cannot store different concrete types in a collection, we have to put them on the heap and store their addresses. -220. Then this flow is what is passed to Peace's common functionality to use in execution or display. - -221. Commands. Commands are one of the common functionality that Peace provides. -222. Given a flow and parameters, it invokes different functions within each item. -223. For example, the Discover command. -224. What is the current state of each item? What is the goal state of each item? -225. The discover command will run these functions, store the state, and display it to the user. -226. The Diff command will compute and show the difference between the current and goal states of each item. -227. The Ensure command will turn the current state of each item, into its goal state, through the apply function. -228. The Clean command is similar, where it turns the current state into the clean state, also through the apply function. -229. So Peace provides common logic to iterate through the items, and call the appropriate functions. -230. and it will also pass the appropriate values between each item. -231. That, is magic. - -232. Going back to the Item definition, besides the functions to read from or write to the item, implementors also have to specify these data types. -233. Input, which we call parameters, and -234. Output, which we call State. -235. The parameters tell the item where to fetch data from, and where to write to, as well as any other information needed to access the item. -236. The state indicates whether or not the item exists, where it lives, and a summary of its contents. -237. This is the type that is returned from the current, goal, and clean state functions. - -238. Putting it all together: -239. We combine the items into a flow, -240. We specify the parameters for each item, -241. Pick an output -- the command line, or web, or both, -242. and these three things together is called a command context. -243. Essentially "all the things you need to run a command". -244. Surface the commands to the user with appropriate names, -245. and this is your tool. - -246. So Peace is a side project, and -247. there are side-side projects that were built in the making of Peace. - -248. The first noticeable one is Interruptible, which adds the ability to interrupt a stream. -249. If you think about playing music, we are streaming bytes to a speaker, and out comes some audio. -250. When we pause the music, the bytes that were buffered still play, but any bytes that were not the audio buffer will not be played. -251. In automation, instead of streaming bytes to a speaker, we are streaming logic, to an executor. -252. When we pause, any logic that was already queued and is executing, will continue to run to completion. -253. Any logic that hasn't been queued, will not be started. -254. So we fully execute what is in progress to completion, and safely stop between steps. -255. And that's how we get safe interruptibility. - -256. The second noticeable project is Dot Interactive. -257. This generates diagrams from structured input. -258. So it takes a data model with the nodes and edges, generates a diagram using GraphViz dot, and adds styles using Tailwind. -259. And that's what I've used for most of the diagrams you've seen today. - -260. Now rounding off, what's the status of Peace? Is it ready to be used? - -261. For development workflows, or short lived environments, where the environment does not live longer than one version of a tool, -262. I'd say it is ready. -263. But for production workflows, or environments that need to be stable, then Peace is not ready. -264. Don't use it, you will not have Peace. - -265. In the table below, you can see the command execution and CLI functionality is stable, -266. The web interface is definitely not stable -- it was hacked together last month for this demo, and -267. the most important one for readiness is API and data stability, which may take me a year to complete. - -268. Links to the project: -269. peace.mk for the project website -270. Slides are on peace.mk/book. -271. github.com/azriel91/peace for the repository. - -272. To wrap up, I'd like to end with this note: -273. To engineer with empathy, -274. whether it is verbal, visual, or vocal, -275. refine your voice, connect, -276. and communicate with clarity. - -277. Thank you for listening, and I'm happy to take questions. +64. You'll see that it picks up where it left off, and completes the process. +65. What you think it should do, it does. No surprises. + +66. So in summary, with information, the goldilocks principle applies: +67. Too much information is overwhelming, too little is not useful, and there's some middle ground which is just right. +68. The Peace framework generally tries to fit the most relevant information on one screen. + +69. The second thing we took away, was control. + +70. Most automation tools give you one button -- start -- and that's it. +71. While pressing start is not difficult, knowing whether the automation will do, what we think it will, is difficult. + +72. What we should understand *before* starting anything, is: +73. Where we are -- our current state, +74. Where we want to go -- our goal state, and +75. The distance between the two. +76. Then we can press start. + +77. But when we press start, and change our mind, can we stop the process? +78. Without automation, we can. + +79. With automation, while pressing Ctrl C on a command line tool is one form of interruption, +80. what we really care about, is safe interruption. +81. If we can interrupt the process, adjust the parameters, press go, and have the automation pick up where it left off, that would be great. +82. And don't undo all of the work you did to get to this point. + +83. Let's see all of this control, in action. +84. Before we run our deployment, what is the current state? `./envman status`. +85. Target state? `./envman goal` +86. What's the difference? `./envman diff` +87. And for interruptibility, when we deploy, we'll stop the process halfway. +88. `./envman deploy`, ctrl c. + +89. Here you can see steps 1 through 3, and step 5 were complete, +90. and step 4 and 6 were not started due to the interruption. +91. If we look at the diff: `./envman diff`, +92. you can see that steps 1, 2, 3, and 5 are done, steps 4 and 6 haven't been executed. +93. If we change our parameters, to using version 0.1.2 instead of 0.1.1 of our web application, +94. the diff will now show that step 1 will change. +95. And if we run deploy again, that is exactly what happens. + +96. When cleaning up, we can also interrupt the process. +97. Steps 1, 4, 5, and 6 were cleaned, and 2 and 3 were not. +98. And we can choose to either deploy the environment again, or clean up fully. +99. Let's deploy it to completion. `deploy`, `clean`. + +100. Morale. +101. Not everyone who uses automation tools has a software background, and not everyone uses the command line all the time. +102. So why not create something that caters for these situations as well? + +103. Back to understandability, normally when explaining what automation does, +104. we tend to draw a diagram on the whiteboard, +105. or create a diagram in an internal documentation site. +106. However, it's never really accurate, and it's usually a tangle of overlapping boxes and lines, +107. so it is hard to understand, because the information isn't clear. + +108. So here's a web interface. `./envman web` +109. Based on the code written for your automation, two diagrams are generated: +110. The one on the left is called the Progress diagram, which shows the steps in your process, +111. and the one on the right is the Outcome diagram, which shows what the deployed environment looks like, before you deploy it. +112. By clicking on these steps on the right, we get to see what is happening in that step. + +113. All of this is generated from your automation code. Magic. +114. This is what you can use to teach someone, or self learn, what the automation process is, and what the environment looks like. +115. And you don't have to keep erasing and redrawing lines on the whiteboard. +116. Which step was unclear? This one? Let's go through that again. + +117. Now, this is great, but I like this one. +118. The diagram you saw is the example environment, but what does the actual environment look like? +119. We can discover it. +120. The diagram on the right has faded boxes for each resource, indicating that it doesn't exist. +121. When I click deploy, you can watch the progress diagram on the left, which will show you which steps are being executed, +122. or you can watch the outcome diagram on the right, which will show you the interactions between hosts, that are happening in real time. + +123. All of the resources have been created, so they are now visible. +124. When we clean up, the boxes in the diagram become faded again. + +125. And if we were to have an error, as we did before, we should see it clearly. +126. *slow down internet, click deploy* +127. Hey look it's gone red. +128. So very quickly, from the user interface, you can tell which step the error came from, +129. as well as which resources it involves. +130. And we can surface the timeout message on the web interface, I just haven't coded that part yet. +131. Cool. + +132. So for morale, a lot of effort has been put into aesthetics. +133. For seeing the state of the system, showing one line for each resource, with a link to the full detail, is deliberate. +134. If you've ever been on-call and gotten a call out in the middle of the night, it's very annoying to have to go and find each resource that is part of the system you are investigating. +135. If I can think it, take me there. + +136. For progress, we present the information at a level of detail that is digestable, + +137. and for errors, instead of panicking, which is visually equivalent of printing a stack trace, +138. we take that error, refine it, and make it beautiful. +139. Always include what went wrong, the reason, and how to recover, +140. because when help people recover from a bad situation, +141. you recover their morale. + +142. With all of these aesthetic refinements, that box, is no longer opaque. +143. It is completely, clear. +144. You can see inside it, you can understand it, and you can control it. + +145. How does all of this work? +146. Magic. + +147. Architecture, how does it fit together? + +148. The Peace framework is categorised into two main parts. +149. The item definition, which is the common shape of logic and data, for anything that is managed by automation, and +150. Common functionality, which works with those items to provide command execution and a user interface. +151. Item crates contain the logic and data to automate one thing, and +152. a tool is the thing that connects items together and passes them to the Peace framework. +153. These groupings are deliberate, so that you can share and reuse common automation logic, +154. while keeping proprietary values and workflows within your tool. +155. Let's go deeper. + +156. If you think of one step in a process, normally we would write code to *do* the step. +157. But instead of only writing code that does work, +158. we also have functions that fetch information about the step. +159. What is the current state of the thing I'm managing? +160. What will it be, after the automation logic is executed? +161. What's the difference between these states? +162. What does it look like if it's not there? +163. Essentially, functions to show me what it is and what it will be, without changing anything. + +164. A collection of functions is called an Item. +165. And a collection of items, is called a Flow. +166. And a flow also contains the dependency ordering between items. +167. Then this flow is passed to Peace's to execute or display information. + +168. Commands. Commands are one of the common functionality that Peace provides. +169. Given a flow and parameters, it invokes different functions within each item. +170. For example, the Discover command will run these functions, store the state, and display it to the user. +171. The Diff command will compute and show the difference between the current and goal states of each item. +172. The Ensure command will turn the current state of each item, into its goal state, through the apply function. +173. The Clean command is similar, where it turns the current state into the clean state, also through the apply function. +174. So Peace provides common logic to iterate through the items, and call the appropriate functions. +175. and it will also pass the appropriate values between each item. +176. That, is magic. + +177. Putting it all together: +178. We combine the items into a flow, +179. We specify the parameters for each item, +180. Pick an output -- the command line, or web, or both, +181. and call the right command. +182. Surface the commands to the user with appropriate names, +183. and this is your tool. + +184. Now rounding off, what's the status of Peace? Is it ready to be used? + +185. For development workflows, or short lived environments, where the environment does not live longer than one version of a tool, +186. It's usable. +187. But for production workflows, or environments that need to be stable, then Peace is not ready. +188. Don't use it, you will not have Peace. + +189. Links to the project: +190. peace.mk for the project website +191. Slides are on peace.mk/book. +192. github.com/azriel91/peace for the repository. + +193. To wrap up, I'd like to end with this note: +194. To engineer with empathy, +195. whether it is verbal, visual, or vocal, +196. refine your voice, connect, +197. and communicate with clarity. + +198. Thank you for listening, and I'm happy to take questions. diff --git a/doc/src/learning_material/peace_zero_stress_automation/envman_example_tool.md b/doc/src/learning_material/peace_zero_stress_automation/envman_example_tool.md index fdd52482..693ac0a9 100644 --- a/doc/src/learning_material/peace_zero_stress_automation/envman_example_tool.md +++ b/doc/src/learning_material/peace_zero_stress_automation/envman_example_tool.md @@ -1,6 +1,13 @@ # 🏗️ envman (example tool)
+
+ 💻
+

Code

+
+
+ + +

Peace

-

Constraints and
common functionality

-
-
- + -
-
- 💻
-

Code

-

Logic and data types

= @@ -27,7 +25,6 @@
⚙️

envman

-

Automation software

diff --git a/doc/src/learning_material/peace_zero_stress_automation/origin_story/automation.md b/doc/src/learning_material/peace_zero_stress_automation/origin_story/automation.md index f4b09bd9..96f76d1e 100644 --- a/doc/src/learning_material/peace_zero_stress_automation/origin_story/automation.md +++ b/doc/src/learning_material/peace_zero_stress_automation/origin_story/automation.md @@ -7,5 +7,5 @@ width="250" />
-source +source
diff --git a/doc/src/learning_material/peace_zero_stress_automation/origin_story/automation/automation.svg b/doc/src/learning_material/peace_zero_stress_automation/origin_story/automation/automation.svg index 821bd08d..3da2ead7 100644 --- a/doc/src/learning_material/peace_zero_stress_automation/origin_story/automation/automation.svg +++ b/doc/src/learning_material/peace_zero_stress_automation/origin_story/automation/automation.svg @@ -3,13 +3,13 @@ - + - + G a - -💻 - Prep Servers -(automatic) + +💻 +  Prep Servers +- +(manual) ++(automatic) b - -📥 - Copy Software -(automatic) + +📥 +  Copy Software +- +(manual) ++(automatic) a->b - - + + c - -💽 - Install -(automatic) + +💽 +  Install +- +(manual) ++(automatic) b->c - - + + d - -📝 - Configure -(automatic) + +📝 +  Configure +- +(manual) ++(automatic) c->d - - + + e - -👷 - Test -(automatic) + +👷 +  Test +- +(manual) ++(automatic) d->e - - + + diff --git a/doc/src/learning_material/peace_zero_stress_automation/peace_framework.md b/doc/src/learning_material/peace_zero_stress_automation/peace_framework.md index 1231ff12..d2144426 100644 --- a/doc/src/learning_material/peace_zero_stress_automation/peace_framework.md +++ b/doc/src/learning_material/peace_zero_stress_automation/peace_framework.md @@ -1,63 +1,48 @@ # 🕊️ Peace Framework
-
- Peace logo
-

Peace

-

Constraints and
Common functionality

-
-
- + -
💻
-

Code

-

Logic and data types

+

Work ×1

+

Scripts / Code

- = + ⇒
⚙️
-

Nice Tool

-

Automation software

+

Value ×1

+

Automation software
(basic)

-
-
- ⚙️
-

Nice Tool

-

Automation software

+ 💻
+

Work ×1

+

Code

+
- 🔡
-

Data

-

Values for parameters

+ Peace logo
+

Peace

+

Common functionality
(UI, interruptibility, etc.)

- = + ⇒
- 🚀
-

Work Done

-

App built and published

-

Environment deployed

+ ⚙️
+

Value ×10

+

Automation software
(really good)

- -