v2.1 Release (#76)

* linkedin example readability
* scrollTo() added 2nd param to set scroll wait time
- 2nd param is optional will fallback value 2500 (2.5seconds)
* rename LinkedIn "articles" to "user posts"
- linkedin differntiates between posts and articles by a single user, so
  what the function has been targetting is actually user posts, not user
  articles
* polish
* update for v2.1
- 2 new botaction types
* standardize examples/linkedin cli command
* webpack build update for 2.1
- includes scrapers and abort botactions
- includes new helpers
- tested locally with separate project against /dist
* reduce duplicated code with vars

README.md

@@ -17,22 +17,20 @@ Why choose Botmation?
 
 <img alt="Baby Bot" src="https://raw.githubusercontent.com/mrWh1te/Botmation/master/assets/art/baby_bot.PNG" width="125" align="right">
 
-It empowers Puppeteer code with a simple pattern that maximizes code readability, reusability and testability.
+It empowers Puppeteer code with a simple pattern to maximize your code readability, reusability and testability.
 
 It has a compositional design with safe defaults for building bots with less code.
 
-It encourages learning at your own pace, to inspire an appreciation for the possibilities of Functional programming.
+It encourages a learn at your own pace approach to exploring the possibilities of Functional programming.
 
-It has 100% library code test coverage.
+It has 100% source code test coverage.
 
 # Introduction
 
 [Botmation](https://botmation.dev) is simple functional framework for [Puppeteer](https://github.com/puppeteer/puppeteer) to build online Bots in a composable, testable, and declarative way. It provides a simple pattern focused on a single type of function called `BotAction`. 
 
 `BotAction`'s handle almost everything from simple tasks in crawling and scraping the web to logging in & automating your social media. They are composable. They make assembling Bots easy, declarative, and simple.
 
-You can compose new `BotAction`'s from ones provided or build your own from scratch, then mix them up.
-
 The possibilities are endless!
 
 # Getting Started
@@ -69,10 +67,12 @@ After intalling through `npm`, import any `BotAction` from the main module:
 import { chain, goTo, screenshot } from 'botmation'
 ```
 
-As of v2.0.x, there are 12 groups of `BotAction` to compose with: 
+As of v2.1.x, there are 14 groups of `BotAction` to compose from: 
 
 <img alt="Leader Bot" src="https://raw.githubusercontent.com/mrWh1te/Botmation/master/assets/art/red_bot.PNG" width="200" align="right" style="position: relative;top: 30px;">
 
+ - [abort](https://www.botmation.dev/api/abort)
+    - abort an assembly of `BotAction`'s
  - [assembly-line](https://www.botmation.dev/api/assembly-lines)
     - compose and run `BotAction`'s in lines
  - [console](https://www.botmation.dev/api/console)
@@ -93,6 +93,10 @@ As of v2.0.x, there are 12 groups of `BotAction` to compose with:
     - read/write/delete from a page's Local Storage
  - [navigation](https://www.botmation.dev/api/navigation)
     - change the page's URL, wait for form submissions to change page URL, back, forward, refresh
+ - [pipe](https://www.botmation.dev/api/pipe)
+    - functions specific to Piping
+ - [scrapers](https://www.botmation.dev/api/scrapers)
+    - scrape HTML documents with an HTML parser and evaluate JavaScript inside a Page
  - [utilities](https://www.botmation.dev/api/utilties)
     - handle more complex logic patterns ie if statements and for loops
 
@@ -105,6 +109,7 @@ In the `./src/examples` [directory](/src/examples) of this repo (excluded from t
  - [Generate Screenshots](/src/examples/screenshots.ts)
  - [Save a PDF](/src/examples/pdf.ts)
  - [Instagram Login](/src/examples/instagram.ts)
+ - [LinkedIn Like Feed Posts](/src/examples/linkedin.ts)
 
 # Dev Notes
 

package.json

@@ -21,10 +21,9 @@
     "examples/puppeteer-cluster": "node build/examples/puppeteer-cluster.js",
     "examples/screenshots": "node build/examples/screenshots.js",
     "examples/pdf": "node build/examples/screenshots.js",
-    "linkedin": "node build/examples/linkedin.js",
+    "examples/linkedin": "node build/examples/linkedin.js",
     "test": "jest --runInBand",
-    "localtestsite": "http-server src/tests/server",
-    "insta": "npm run examples/instagram"
+    "localtestsite": "http-server src/tests/server"
   },
   "keywords": [
     "social",

src/botmation/actions/assembly-lines.ts

@@ -113,7 +113,7 @@ export const pipe =
  * Before each assembled BotAction is ran, the pipe is switched back to whatever is set `toPipe`
  * `toPipe` is optional and can be provided by an injected pipe object value (if nothing provided, default is undefined)
  * 
- *  AbortLineSignal default abort(1) is ignored until a MatchesSignal is returned by an assembled BotAction, marking that at least one Case has ran
+ *  AbortLineSignal default abort(1) is ignored until a CasesSignal is returned by an assembled BotAction, marking that at least one Case has ran
  *    to break that, you can abort(2+) 
  *  This is to support the classic switch/case/break flow where its switchPipe/pipeCase/abort
  *    Therefore, if a pipeCase() does run, its returning MatcheSignal will be recognized by switchPipe and then lower the required abort count by 1
@@ -144,7 +144,7 @@ export const switchPipe =
           let resolvedActionResult = await action(page, ...injects)
 
           // resolvedActionResult can be of 3 things
-          // 1. MatchesSignal 2. AbortLineSignal 3. PipeValue
+          // 1. CasesSignal 2. AbortLineSignal 3. PipeValue
           // switchPipe will return (if not aborted) an array of all the resolved results of each BotAction assembled in the switchPipe()() 2nd call
           if (isCasesSignal(resolvedActionResult) && resolvedActionResult.conditionPass) {
             hasAtLeastOneCaseMatch = true

src/botmation/actions/cookies.ts

@@ -4,7 +4,7 @@ import { BotFilesAction } from '../interfaces/bot-actions'
 import { enrichBotFileOptionsWithDefaults } from '../helpers/files'
 import { BotFileOptions } from '../interfaces'
 import { getFileUrl } from '../helpers/files'
-import { unpipeInjects } from 'botmation/helpers/pipe'
+import { unpipeInjects } from '../helpers/pipe'
 
 /**
  * @description   Parse page's cookies and save them as JSON in a local file

src/botmation/actions/navigation.ts

@@ -69,9 +69,10 @@ export const wait = (milliseconds: number): BotAction => async() => {
 /**
  * 
  * @param htmlSelector 
+ * @param waitTimeForScroll milliseconds to wait for scrolling
  */
-export const scrollTo = (htmlSelector: string): BotAction => 
+export const scrollTo = (htmlSelector: string, waitTimeForScroll: number = 2500): BotAction => 
   chain(
-    evaluate(scrollToElement, htmlSelector),
-    wait(2500) // wait for scroll to complete
+    evaluate(scrollToElement, htmlSelector), // init's scroll code, but does not wait for it to complete
+    wait(waitTimeForScroll) // wait for scroll to complete
   )

src/botmation/actions/pipe.ts

@@ -37,10 +37,10 @@ export const emptyPipe: BotAction = async () => undefined
  * Similar to givenThat except instead of evaluating a BotAction for TRUE, its testing an array of values against the pipe object value for truthy.
  * A value can be a function. In this case, the function is treated as a callback, expected to return a truthy expression, is passed in the pipe object's value
  * @param valuesToTest 
- * @return AbortLineSignal|MatchesSignal
- *  If no matches are found or matches are found, a MatchesSignal is returned
+ * @return AbortLineSignal|CasesSignal
+ *  If no matches are found or matches are found, a CasesSignal is returned
  *  It is determined if signal has matches by using hasAtLeastOneMatch() helper
- *  If assembled BotAction aborts(1), it breaks line & returns a MatchesSignal with the matches
+ *  If assembled BotAction aborts(1), it breaks line & returns a CasesSignal with the matches
  *  If assembled BotAction aborts(2), it breaks line & returns AbortLineSignal.pipeValue
  *  If assembled BotAction aborts(3+), it returns AbortLineSignal(2-)
  */
@@ -86,7 +86,7 @@ export const pipeCase =
       }
 
 /**
- * runs assembled actions ONLY if ALL cases pass otherwise it breaks the case checking and immediately returns an empty MatchesSignal
+ * runs assembled actions ONLY if ALL cases pass otherwise it breaks the case checking and immediately returns an empty CasesSignal
  * it's like if (case && case && case ...)
  * Same AbortLineSignal behavior as pipeCase()()
  * @param valuesToTest 

src/botmation/actions/scrapers.ts

@@ -81,7 +81,7 @@ export const $$ = <R = CheerioStatic[]>(htmlSelector: string, higherOrderHTMLPar
 
 /**
  * Evaluate functions inside the `page` context
- * @param functionToEvaluate 
+ * @param functionToEvaluate
  * @param functionParams 
  */
 export const evaluate = (functionToEvaluate: EvaluateFn<any>, ...functionParams: any[]): BotAction<any> => 

src/botmation/index.ts

@@ -30,12 +30,14 @@ export * from './actions/utilities'
 //
 // Helpers
 export * from './helpers/abort'
+export * from './helpers/cases'
 export * from './helpers/console'
 export * from './helpers/files'
 export * from './helpers/indexed-db'
 export * from './helpers/local-storage'
 export * from './helpers/navigation'
 export * from './helpers/pipe'
+export * from './helpers/scrapers'
 
 //
 // Class

src/botmation/sites/linkedin/actions/feed.ts

@@ -21,7 +21,7 @@ import {
   postIsPromotion,
   postIsJobPostings,
   postIsUserInteraction,
-  postIsUserArticle,
+  postIsUserPost,
   postIsAuthoredByAPerson
 } from '../helpers/feed'
 
@@ -39,13 +39,13 @@ export const scrapeFeedPost = (postDataId: string): BotAction<CheerioStatic> =>
   $('.application-outlet .feed-outlet [role="main"] [data-id="'+ postDataId + '"]')
 
 /**
- * If the post hasn't been populated (waits loading), then scroll to it to trigger lazy loading then scrape it to return the hydrated version of it
+ * If the post hasn't been populated (waits loading), then scroll to it to cause lazy loading then scrape it to return the hydrated version of it
  * @param post 
  */
-export const ifPostNotLoadedTriggerLoadingThenScrape = (post: CheerioStatic): BotAction<CheerioStatic> =>
+export const ifPostNotLoadedCauseLoadingThenScrape = (post: CheerioStatic): BotAction<CheerioStatic> =>
   // linkedin lazily loads off screen posts, so check beforehand, and if not loaded, scroll to it, then scrape it again
   pipe(post)(
-    errors('LinkedIn triggerLazyLoadingThenScrapePost()')(
+    errors('LinkedIn causeLazyLoadingThenScrapePost()')(
       pipeCase(postHasntFullyLoadedYet)(
         scrollTo('.application-outlet .feed-outlet [role="main"] [data-id="'+ post('[data-id]').attr('data-id') + '"]'),
         scrapeFeedPost(post('[data-id]').attr('data-id') + '')
@@ -62,7 +62,7 @@ export const ifPostNotLoadedTriggerLoadingThenScrape = (post: CheerioStatic): Bo
  *         It would be nice to rely on ie Post.id as param to then find that "Like" button in page to click. In order to, de-couple this function
  * @param post 
  */
-export const likeArticle = (post: CheerioStatic): BotAction =>
+export const likeUserPost = (post: CheerioStatic): BotAction =>
   // Puppeteer.page.click() returned promise will reject if the selector isn't found
   //    so if button is Pressed, it will reject since the aria-label value will not match
   errors('LinkedIn like() - Could not Like Post: Either already Liked or button not found')(
@@ -76,12 +76,12 @@ export const likeArticle = (post: CheerioStatic): BotAction =>
  * @description   Demonstration of what's currently possible, this function goes beyond the scope of its name, but to give an idea on how something more complex could be handled
  * @param peopleNames 
  */
-export const likeArticlesFrom = (...peopleNames: string[]): BotAction => 
+export const likeUserPostsFrom = (...peopleNames: string[]): BotAction => 
   pipe()(
     scrapeFeedPosts,
     forAll()(
       post => pipe(post)(
-        ifPostNotLoadedTriggerLoadingThenScrape(post),
+        ifPostNotLoadedCauseLoadingThenScrape(post),
         switchPipe()(
           pipeCase(postIsPromotion)(
             map((promotionPost: CheerioStatic) => promotionPost('[data-id]').attr('data-id')),
@@ -98,11 +98,11 @@ export const likeArticlesFrom = (...peopleNames: string[]): BotAction =>
             log(`Followed User's Interaction (ie like, comment, etc)`)
           ),
           abort(),
-          pipeCase(postIsUserArticle)(
+          pipeCase(postIsUserPost)(
             pipeCase(postIsAuthoredByAPerson(...peopleNames))(
               // scroll to post necessary to click off page link? ie use scrollTo() "navigation" BotAction
               // the feature, auto-scroll, was added to `page.click()` in later Puppeteer version, irc
-              likeArticle(post),
+              likeUserPost(post),
               log('User Article "liked"')
             ),
             emptyPipe,

src/botmation/sites/linkedin/helpers/feed.ts

@@ -5,7 +5,7 @@ import { feedPostAuthorSelector } from '../selectors'
  * User articles are posts created by users you're either connected too directly (1st) or are following
  * @param peopleNames 
  */
-export const postIsUserArticle: ConditionalCallback<CheerioStatic> = (post: CheerioStatic) => {
+export const postIsUserPost: ConditionalCallback<CheerioStatic> = (post: CheerioStatic) => {
   const sharedActorFeedSupplementaryInfo = post('.feed-shared-actor__supplementary-actor-info').text().trim().toLowerCase()
 
   return sharedActorFeedSupplementaryInfo.includes('1st') || sharedActorFeedSupplementaryInfo.includes('following')

src/examples/linkedin.ts

@@ -2,15 +2,15 @@ import puppeteer from 'puppeteer'
 
 // General BotAction's
 import { log } from 'botmation/actions/console'
-// import { screenshot } from 'botmation/actions/files'
+import { screenshot } from 'botmation/actions/files'
 import { loadCookies } from 'botmation/actions/cookies'
 
 // More advanced BotAction's
 import { pipe, saveCookies, wait, errors, givenThat } from 'botmation'
 import { login, isGuest, isLoggedIn } from 'botmation/sites/linkedin/actions/auth'
 import { toggleMessagingOverlay } from 'botmation/sites/linkedin/actions/messaging'
-import { likeArticlesFrom } from 'botmation/sites/linkedin/actions/feed'
-import { goHome } from 'botmation/sites/linkedin/actions/navigation'
+import { likeUserPostsFrom } from 'botmation/sites/linkedin/actions/feed'
+import { goHome, goToFeed } from 'botmation/sites/linkedin/actions/navigation'
 
 // Helper for creating filenames that sort naturally
 const generateTimeStamp = (): string => {
@@ -53,10 +53,12 @@ const generateTimeStamp = (): string => {
     wait(5000), // tons of stuff loads... no rush
 
     givenThat(isLoggedIn)(
+      goToFeed,
+
       toggleMessagingOverlay, // by default, Messaging Overlay loads in open state
-      // screenshot(generateTimeStamp()), // filename ie "2020-8-21-13-20.png"
+      screenshot(generateTimeStamp()), // filename ie "2020-8-21-13-20.png"
 
-      likeArticlesFrom('Peter Parker', 'Harry Potter')
+      likeUserPostsFrom('Peter Parker', 'Harry Potter')
     )
    )
 

src/tests/botmation/actions/navigation.spec.ts

@@ -121,6 +121,11 @@ describe('[Botmation] actions/navigation', () => {
 
     expect(mockScrollToElement).toHaveBeenNthCalledWith(1, 'some-element-far-away')
     expect(mockSleep).toHaveBeenNthCalledWith(2, 2500)
+
+    await scrollTo('some-element-far-far-away', 5000)(mockPage)
+
+    expect(mockScrollToElement).toHaveBeenNthCalledWith(2, 'some-element-far-far-away')
+    expect(mockSleep).toHaveBeenNthCalledWith(3, 5000)
   })
 
   // clean up

tsconfig.dist.json

@@ -11,7 +11,8 @@
     "baseUrl": "./src",
     "sourceMap": true,
     "esModuleInterop": true,
-    "declaration": true
+    "declaration": true,
+    "downlevelIteration": true
   },
   "exclude": [
     "node_modules",

tsconfig.json

@@ -11,7 +11,8 @@
     "baseUrl": "./src",
     "sourceMap": true,
     "esModuleInterop": true,
-    "declaration": true
+    "declaration": true,
+    "downlevelIteration": true
   },
   "exclude": [
     "assets",