-
Notifications
You must be signed in to change notification settings - Fork 284
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
2b10e2f
commit b8db6f2
Showing
66 changed files
with
7,053 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
const fs = require('fs-extra'); | ||
const path = require('path'); | ||
const crypto = require('crypto'); | ||
|
||
async function generateIndex() { | ||
const recipesDir = path.join(__dirname, '../../docs/recipes'); | ||
const outputPath = path.join(recipesDir, 'index.json'); | ||
const files = await fs.readdir(recipesDir); | ||
const index = []; | ||
|
||
for (const file of files) { | ||
if (file.endsWith('.md')) { | ||
const filePath = path.join(recipesDir, file); | ||
const content = await fs.readFile(filePath, 'utf-8'); | ||
const titleMatch = content.match(/^#\s+(.+)$/m); | ||
const title = titleMatch ? titleMatch[1] : 'Untitled'; | ||
const hash = crypto.createHash('md5').update(content).digest('hex'); | ||
index.push({ | ||
file: file, | ||
title: title, | ||
path: `/docs/recipes/${file}`, | ||
hash: hash, | ||
}); | ||
} | ||
} | ||
|
||
await fs.writeJson(outputPath, index, { spaces: 2 }); | ||
console.log(`Index written to ${outputPath}`); | ||
} | ||
|
||
generateIndex().catch(err => { | ||
console.error(err); | ||
process.exit(1); | ||
}); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
name: Generate Recipes Index | ||
|
||
on: | ||
push: | ||
branches: | ||
- '**' # This will trigger on push to any branch | ||
paths: | ||
- 'docs/recipes/**' | ||
workflow_dispatch: | ||
|
||
jobs: | ||
generate-index: | ||
runs-on: ubuntu-latest | ||
|
||
steps: | ||
- name: Checkout repository | ||
uses: actions/checkout@v2 | ||
|
||
- name: Set up Node.js | ||
uses: actions/setup-node@v2 | ||
with: | ||
node-version: '14' | ||
|
||
- name: Install dependencies | ||
run: | | ||
npm install fs-extra | ||
- name: Generate index | ||
run: | | ||
node .github/scripts/generate-index-recipes.js | ||
- name: Commit and push changes | ||
run: | | ||
BRANCH_NAME=$(echo ${GITHUB_REF#refs/heads/}) | ||
git config --local user.email "action@github.com" | ||
git config --local user.name "GitHub Action" | ||
git add docs/recipes/index.json | ||
git commit -m "Update recipes index" | ||
git push origin $BRANCH_NAME |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,54 @@ | ||
<!-- | ||
{ | ||
"title": "Externalize strings", | ||
"id": "Externalizing_Strings", | ||
"description": "Externalize strings from generated class files to separate files. This method is used to reduce the memory of the static contents for templates.", | ||
"keywords": [ | ||
"Externalize strings", | ||
"Memory reduction", | ||
"Class files", | ||
"Static contents", | ||
"Lucee" | ||
] | ||
} | ||
--> | ||
## Externalize strings ## | ||
|
||
Externalize strings from generated class files to separate files. This method is used to reduce the memory of the static contents for templates. We explain this method with a simple example below: | ||
|
||
**Example:** | ||
|
||
//index.cfm | ||
|
||
```lucee | ||
<cfset year = 1960> | ||
<html> | ||
<body> | ||
<h1>....</h1> | ||
....... | ||
....... | ||
It was popular in the <cfoutput> #year# </cfoutput> | ||
....... | ||
<b>....</b> | ||
</body> | ||
</html> | ||
``` | ||
|
||
1. Here the Index.cfm file contains a lot of strings (static contents), but there is no functionality. The file just gives a cfoutput with year. The variable string 'year' is already declared by using in top of the Index.cfm page. | ||
|
||
2. Execute the CFM page in a browser. A class file is created in the `webapps/ROOT/WEB-INF/lucee/cfclasses/` directory while the CFM file is executed. The run time compiler compiles that file to load the Java bytecode and execute it. | ||
|
||
3. Right click the class file. Then see `Get info`. For example, in my class file there is 8Kb size on the disk. In Lucee, the CFM file with its strings was also loaded. So a lot of memory could be occupied just by string loading the bytecode. To avoid this problem, the Lucee admin has the following solution: | ||
|
||
- Lucee admin --> Language/compiler --> Externalize strings | ||
- This `Externalize strings` setting has four options. Select any one option to test. We selected the fourth option (externalize strings larger than 10 characters). | ||
- Again run the CFM page in a browser. The class file is created with lower memory size than the original 8Kb on disk. | ||
- In addition, it created a text file too. The text file contains the strings from the CFM page. The cfoutput with year is simply not there. The byte code will crop the piece of cfoutput content from the CFM file. | ||
|
||
So, the string 'year' is no longer in memory. When the bytecode is called, it loads the string into memory. The memory is not occupied forever and this reduces the footprint of our application. | ||
|
||
### Footnotes ### | ||
|
||
Here you can see the above details in video | ||
|
||
[Externalize strings](https://youtu.be/AUcsHkVFXHE) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,133 @@ | ||
<!-- | ||
{ | ||
"title": "Query of Queries sometimes it rocks, sometimes it sucks", | ||
"id": "QOQ_Sucks", | ||
"related": [ | ||
"tag-query", | ||
"function-queryexecute", | ||
"function-queryfilter" | ||
], | ||
"categories": [ | ||
"query" | ||
], | ||
"description": "This document explains why Query of Queries (QoQ) may or may not be the best approach for your use case.", | ||
"keywords": [ | ||
"Query of Queries", | ||
"QoQ", | ||
"Performance", | ||
"Query Filter", | ||
"Query Sort", | ||
"Lucee" | ||
] | ||
} | ||
--> | ||
## The good, the bad and the ugly ## | ||
|
||
This document explains why Query of Queries (QoQ) may or may not be the best approach for your use case. | ||
|
||
- **PRO**: It's nice to work with in-memory datasets/queries using SQL. | ||
- **CON**: It can be very slow, depending on the use case. | ||
|
||
Update: The performance of QoQ has been dramatically improved for single tables since 5.3.8! | ||
|
||
[Improving Lucee's Query of Query Support](http://wwvv.codersrevolution.com/blog/improving-lucees-query-of-query-support) | ||
|
||
There has also been a lot of work done to improve the "correctness" of the native SQL engine's behavior. | ||
|
||
[QoQ tickets](https://luceeserver.atlassian.net/issues/?jql=text%20~%20%22qoq%22%20ORDER%20BY%20updated) | ||
|
||
Currently, Lucee QoQ only supports `SELECT` statements; `UPDATE` and `INSERT` aren't yet supported. | ||
|
||
Lucee has two QoQ engines, a fast native engine which only works on a single table. | ||
|
||
Any SQL using multiple tables, i.e., with a JOIN, will fall back to the HSQLDB engine. | ||
|
||
The HSQLDB engine requires loading all the queries into temporary tables and is currently Java synchronized, all of which can affect performance. | ||
|
||
If the native QoQ engine fails on a single table query, by default, Lucee will attempt to fall back on the HSQLDB engine. | ||
|
||
See `LUCEE_QOQ_HSQLDB_DISABLE` and `LUCEE_QOQ_HSQLDB_DEBUG` under [[running-lucee-system-properties]]. | ||
|
||
### Example: ### | ||
|
||
```lucee+trycf | ||
<cfscript> | ||
q = QueryNew("name, description"); | ||
loop times=3 { | ||
getFunctionList().each(function(f){ | ||
var fd = getFunctionData(arguments.f); | ||
var r = QueryAddRow(q); | ||
QuerySetCell(q,"name", fd.name, r); | ||
QuerySetCell(q,"description", fd.description, r); | ||
}); | ||
} | ||
dump(server.lucee.version); | ||
dump(var=q.recordcount, | ||
label="demo data set size"); | ||
s = "the"; | ||
</cfscript> | ||
<cftimer type="outline"> | ||
<cfscript> | ||
q3 = q.filter(function(row){ | ||
return (row.description contains s); | ||
}); | ||
</cfscript> | ||
</cftimer> | ||
<cfdump var=#q3.recordcount#> | ||
``` | ||
|
||
In this example, we have a QOQ with the persons table. | ||
|
||
```luceescript | ||
// index.cfm | ||
directory sort="name" action="list" directory=getDirectoryFromPath(getCurrentTemplatePath()) filter="example.cfm" name="dir"; | ||
loop query=dir { | ||
echo('<a href="#dir.name#">#dir.name#</a><br>'); | ||
} | ||
``` | ||
|
||
```luceescript | ||
// example.cfm | ||
max=1000; | ||
persons=query( | ||
"lastname":["Lebowski","Lebowski","Lebowski","Sobchak"], | ||
"firstname":["Jeffrey","Bunny","Maude","Walter"] | ||
); | ||
// Query of Query | ||
start=getTickCount("micro"); | ||
loop times=max { | ||
query dbtype="query" name="qoq" {echo(" | ||
select * from persons | ||
where lastname='Lebowski' | ||
and firstname='Bunny' | ||
order by lastname | ||
");} | ||
} | ||
dump("Query of Query Execution Time:"&(getTickCount("micro")-start)); | ||
// Query Filter/Sort | ||
start=getTickCount("micro"); | ||
loop times=max { | ||
qf=queryFilter(persons,function (row,cr,qry) {return row.firstname=='Bunny' && row.lastname=='Lebowski';}); | ||
qs=querySort(qf,"lastname"); | ||
} | ||
dump("Query Filter/Sort Execution Time:"&(getTickCount("micro")-start)); | ||
``` | ||
|
||
In this example, we have two different methods of queries. | ||
|
||
1) First one is QoQ. Here, `QoQ` from the `persons` table is executed a thousand times due to the looping required by QoQ. | ||
|
||
2) The second one is calling the function query filter. Query filter filters out the same row the same way the QoQ does. | ||
|
||
3) Execute it in the browser and we get two results (Query of Query execution time and Query filter/sort execution time). Query filter executes at least twice as fast as query of query. Because QoQ loops over and over again, it is slower. If you can avoid QoQ and use the Query filter/sort, your code will execute much faster. | ||
|
||
### Footnotes ### | ||
|
||
You can see these details in the video here: | ||
|
||
[Query of Query Sucks](https://www.youtube.com/watch?v=bUBXzo1WbSM) |
Oops, something went wrong.