Skip to content

Commit

Permalink
add recipes
Browse files Browse the repository at this point in the history
  • Loading branch information
michaeloffner committed Jun 6, 2024
1 parent 2b10e2f commit b8db6f2
Show file tree
Hide file tree
Showing 66 changed files with 7,053 additions and 0 deletions.
34 changes: 34 additions & 0 deletions .github/scripts/generate-index-recipes.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
const fs = require('fs-extra');
const path = require('path');
const crypto = require('crypto');

async function generateIndex() {
const recipesDir = path.join(__dirname, '../../docs/recipes');
const outputPath = path.join(recipesDir, 'index.json');
const files = await fs.readdir(recipesDir);
const index = [];

for (const file of files) {
if (file.endsWith('.md')) {
const filePath = path.join(recipesDir, file);
const content = await fs.readFile(filePath, 'utf-8');
const titleMatch = content.match(/^#\s+(.+)$/m);
const title = titleMatch ? titleMatch[1] : 'Untitled';
const hash = crypto.createHash('md5').update(content).digest('hex');
index.push({
file: file,
title: title,
path: `/docs/recipes/${file}`,
hash: hash,
});
}
}

await fs.writeJson(outputPath, index, { spaces: 2 });
console.log(`Index written to ${outputPath}`);
}

generateIndex().catch(err => {
console.error(err);
process.exit(1);
});
39 changes: 39 additions & 0 deletions .github/workflows/index-recipes.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: Generate Recipes Index

on:
push:
branches:
- '**' # This will trigger on push to any branch
paths:
- 'docs/recipes/**'
workflow_dispatch:

jobs:
generate-index:
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v2

- name: Set up Node.js
uses: actions/setup-node@v2
with:
node-version: '14'

- name: Install dependencies
run: |
npm install fs-extra
- name: Generate index
run: |
node .github/scripts/generate-index-recipes.js
- name: Commit and push changes
run: |
BRANCH_NAME=$(echo ${GITHUB_REF#refs/heads/})
git config --local user.email "action@github.com"
git config --local user.name "GitHub Action"
git add docs/recipes/index.json
git commit -m "Update recipes index"
git push origin $BRANCH_NAME
54 changes: 54 additions & 0 deletions docs/recipes/Externalizing_Strings.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
<!--
{
"title": "Externalize strings",
"id": "Externalizing_Strings",
"description": "Externalize strings from generated class files to separate files. This method is used to reduce the memory of the static contents for templates.",
"keywords": [
"Externalize strings",
"Memory reduction",
"Class files",
"Static contents",
"Lucee"
]
}
-->
## Externalize strings ##

Externalize strings from generated class files to separate files. This method is used to reduce the memory of the static contents for templates. We explain this method with a simple example below:

**Example:**

//index.cfm

```lucee
<cfset year = 1960>
<html>
<body>
<h1>....</h1>
.......
.......
It was popular in the <cfoutput> #year# </cfoutput>
.......
<b>....</b>
</body>
</html>
```

1. Here the Index.cfm file contains a lot of strings (static contents), but there is no functionality. The file just gives a cfoutput with year. The variable string 'year' is already declared by using in top of the Index.cfm page.

2. Execute the CFM page in a browser. A class file is created in the `webapps/ROOT/WEB-INF/lucee/cfclasses/` directory while the CFM file is executed. The run time compiler compiles that file to load the Java bytecode and execute it.

3. Right click the class file. Then see `Get info`. For example, in my class file there is 8Kb size on the disk. In Lucee, the CFM file with its strings was also loaded. So a lot of memory could be occupied just by string loading the bytecode. To avoid this problem, the Lucee admin has the following solution:

- Lucee admin --> Language/compiler --> Externalize strings
- This `Externalize strings` setting has four options. Select any one option to test. We selected the fourth option (externalize strings larger than 10 characters).
- Again run the CFM page in a browser. The class file is created with lower memory size than the original 8Kb on disk.
- In addition, it created a text file too. The text file contains the strings from the CFM page. The cfoutput with year is simply not there. The byte code will crop the piece of cfoutput content from the CFM file.

So, the string 'year' is no longer in memory. When the bytecode is called, it loads the string into memory. The memory is not occupied forever and this reduces the footprint of our application.

### Footnotes ###

Here you can see the above details in video

[Externalize strings](https://youtu.be/AUcsHkVFXHE)
133 changes: 133 additions & 0 deletions docs/recipes/QOQ_Sucks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
<!--
{
"title": "Query of Queries sometimes it rocks, sometimes it sucks",
"id": "QOQ_Sucks",
"related": [
"tag-query",
"function-queryexecute",
"function-queryfilter"
],
"categories": [
"query"
],
"description": "This document explains why Query of Queries (QoQ) may or may not be the best approach for your use case.",
"keywords": [
"Query of Queries",
"QoQ",
"Performance",
"Query Filter",
"Query Sort",
"Lucee"
]
}
-->
## The good, the bad and the ugly ##

This document explains why Query of Queries (QoQ) may or may not be the best approach for your use case.

- **PRO**: It's nice to work with in-memory datasets/queries using SQL.
- **CON**: It can be very slow, depending on the use case.

Update: The performance of QoQ has been dramatically improved for single tables since 5.3.8!

[Improving Lucee's Query of Query Support](http://wwvv.codersrevolution.com/blog/improving-lucees-query-of-query-support)

There has also been a lot of work done to improve the "correctness" of the native SQL engine's behavior.

[QoQ tickets](https://luceeserver.atlassian.net/issues/?jql=text%20~%20%22qoq%22%20ORDER%20BY%20updated)

Currently, Lucee QoQ only supports `SELECT` statements; `UPDATE` and `INSERT` aren't yet supported.

Lucee has two QoQ engines, a fast native engine which only works on a single table.

Any SQL using multiple tables, i.e., with a JOIN, will fall back to the HSQLDB engine.

The HSQLDB engine requires loading all the queries into temporary tables and is currently Java synchronized, all of which can affect performance.

If the native QoQ engine fails on a single table query, by default, Lucee will attempt to fall back on the HSQLDB engine.

See `LUCEE_QOQ_HSQLDB_DISABLE` and `LUCEE_QOQ_HSQLDB_DEBUG` under [[running-lucee-system-properties]].

### Example: ###

```lucee+trycf
<cfscript>
q = QueryNew("name, description");
loop times=3 {
getFunctionList().each(function(f){
var fd = getFunctionData(arguments.f);
var r = QueryAddRow(q);
QuerySetCell(q,"name", fd.name, r);
QuerySetCell(q,"description", fd.description, r);
});
}
dump(server.lucee.version);
dump(var=q.recordcount,
label="demo data set size");
s = "the";
</cfscript>
<cftimer type="outline">
<cfscript>
q3 = q.filter(function(row){
return (row.description contains s);
});
</cfscript>
</cftimer>
<cfdump var=#q3.recordcount#>
```

In this example, we have a QOQ with the persons table.

```luceescript
// index.cfm
directory sort="name" action="list" directory=getDirectoryFromPath(getCurrentTemplatePath()) filter="example.cfm" name="dir";
loop query=dir {
echo('<a href="#dir.name#">#dir.name#</a><br>');
}
```

```luceescript
// example.cfm
max=1000;
persons=query(
"lastname":["Lebowski","Lebowski","Lebowski","Sobchak"],
"firstname":["Jeffrey","Bunny","Maude","Walter"]
);
// Query of Query
start=getTickCount("micro");
loop times=max {
query dbtype="query" name="qoq" {echo("
select * from persons
where lastname='Lebowski'
and firstname='Bunny'
order by lastname
");}
}
dump("Query of Query Execution Time:"&(getTickCount("micro")-start));
// Query Filter/Sort
start=getTickCount("micro");
loop times=max {
qf=queryFilter(persons,function (row,cr,qry) {return row.firstname=='Bunny' && row.lastname=='Lebowski';});
qs=querySort(qf,"lastname");
}
dump("Query Filter/Sort Execution Time:"&(getTickCount("micro")-start));
```

In this example, we have two different methods of queries.

1) First one is QoQ. Here, `QoQ` from the `persons` table is executed a thousand times due to the looping required by QoQ.

2) The second one is calling the function query filter. Query filter filters out the same row the same way the QoQ does.

3) Execute it in the browser and we get two results (Query of Query execution time and Query filter/sort execution time). Query filter executes at least twice as fast as query of query. Because QoQ loops over and over again, it is slower. If you can avoid QoQ and use the Query filter/sort, your code will execute much faster.

### Footnotes ###

You can see these details in the video here:

[Query of Query Sucks](https://www.youtube.com/watch?v=bUBXzo1WbSM)
Loading

0 comments on commit b8db6f2

Please sign in to comment.