-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathhow-to-run-celery-as-daemon.html
264 lines (215 loc) · 15.9 KB
/
how-to-run-celery-as-daemon.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
<!DOCTYPE html>
<html lang="en">
<head>
<link href='//fonts.googleapis.com/css?family=Source+Sans+Pro:300,400,700,400italic' rel='stylesheet' type='text/css'>
<link rel="stylesheet" type="text/css" href="http://www.emadmokhtar.com/theme/stylesheet/style.min.css">
<link rel="stylesheet" type="text/css" href="http://www.emadmokhtar.com/theme/stylesheet/pygments.min.css">
<link rel="stylesheet" type="text/css" href="http://www.emadmokhtar.com/theme/stylesheet/font-awesome.min.css">
<link href="http://www.emadmokhtar.com/feeds/all.atom.xml" type="application/atom+xml" rel="alternate" title="Emad Mokhtar's Framework Atom">
<link href="http://www.emadmokhtar.com/feeds/all.rss.xml" type="application/rss+xml" rel="alternate" title="Emad Mokhtar's Framework RSS">
<link rel="shortcut icon" href="http://www.emadmokhtar.com/images/favicon.ico" type="image/x-icon">
<link rel="icon" href="http://www.emadmokhtar.com/images/favicon.ico" type="image/x-icon">
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="robots" content="" />
<meta name="author" content="Emad Mokhtar" />
<meta name="description" content="Install upstart script Permissions We need to make the user which will be used to run celery an owner of log and run directories. sudo chown -R root:root /var/log/celery/ sudo chown -R root:root /var/run/celery/ Celery upstart 1- Download celeryd file from Celery official github repository. 2- Copy celeryd to /etc/init.d/celeryd sudo cp celeryd /etc/init.d/ 3- Make it executable sudo chmod +x celeryd 4- Make it run on startup sudo update-rc.d celeryd defaults sudo update-rc.d celeryd enable 5- make it owned by root sudo chown root:root celeryd 6- Done, now you can start/stop/restart celeryd as service sudo service celeryd start sudo service celeryd stop sudo service celeryd restart Celerybeat upstart 1- Download celerybeat file from Celery official github repository. 2- Copy celerybeat to /etc/init.d/celerybeat sudo cp celerybeat /etc/init.d/ 3- Make it executable sudo chmod +x celerybeat 4- Make it run on startup sudo update-rc.d celerybeat defaults sudo update-rc.d celerybeat enable 5- make it owned by root sudo chown root:root celerybeat 6- Done, now you can start/stop/restart celerd as service sudo service celerybeat start sudo service …" />
<meta name="keywords" content="django, python, celery">
<meta property="og:site_name" content="Emad Mokhtar's Framework"/>
<meta property="og:title" content="How to run Celery as daemon?"/>
<meta property="og:description" content="Install upstart script Permissions We need to make the user which will be used to run celery an owner of log and run directories. sudo chown -R root:root /var/log/celery/ sudo chown -R root:root /var/run/celery/ Celery upstart 1- Download celeryd file from Celery official github repository. 2- Copy celeryd to /etc/init.d/celeryd sudo cp celeryd /etc/init.d/ 3- Make it executable sudo chmod +x celeryd 4- Make it run on startup sudo update-rc.d celeryd defaults sudo update-rc.d celeryd enable 5- make it owned by root sudo chown root:root celeryd 6- Done, now you can start/stop/restart celeryd as service sudo service celeryd start sudo service celeryd stop sudo service celeryd restart Celerybeat upstart 1- Download celerybeat file from Celery official github repository. 2- Copy celerybeat to /etc/init.d/celerybeat sudo cp celerybeat /etc/init.d/ 3- Make it executable sudo chmod +x celerybeat 4- Make it run on startup sudo update-rc.d celerybeat defaults sudo update-rc.d celerybeat enable 5- make it owned by root sudo chown root:root celerybeat 6- Done, now you can start/stop/restart celerd as service sudo service celerybeat start sudo service …"/>
<meta property="og:locale" content="en_US"/>
<meta property="og:url" content="http://www.emadmokhtar.com/how-to-run-celery-as-daemon.html"/>
<meta property="og:type" content="article"/>
<meta property="article:published_time" content="2016-09-20 15:00:00+03:00"/>
<meta property="article:modified_time" content=""/>
<meta property="article:author" content="http://www.emadmokhtar.com/author/emad-mokhtar.html">
<meta property="article:section" content="Django"/>
<meta property="article:tag" content="django"/>
<meta property="article:tag" content="python"/>
<meta property="article:tag" content="celery"/>
<meta property="og:image" content="{static}/images/celerysmoothie.jpg">
<title>Emad Mokhtar's Framework – How to run Celery as daemon?</title>
</head>
<body>
<aside>
<div>
<a href="http://www.emadmokhtar.com">
<img src="http://www.emadmokhtar.com/images/profile.jpg" alt="" title="">
</a>
<h1><a href="http://www.emadmokhtar.com"></a></h1>
<p>Geek developer who's in search of code perfection.</p>
<nav>
<ul class="list">
<li><a href="http://www.emadmokhtar.com">Blog</a></li>
<li><a href="http://www.emadmokhtar.com/pages/podcasts.html#podcasts">Podcasts</a></li>
<li><a href="http://www.emadmokhtar.com/pages/projects.html#projects">Projects</a></li>
<li><a href="https://emadmokhtar.github.io/resume" target="_blank">Resume</a></li>
</ul>
</nav>
<ul class="social">
<li><a class="sc-envelope-o" href="mailto:emad@emadmokhtar.com" target="_blank"><i class="fa fa-envelope-o"></i></a></li>
<li><a class="sc-github" href="https://www.github.com/EmadMokhtar" target="_blank"><i class="fa fa-github"></i></a></li>
<li><a class="sc-linkedin" href="https://www.linkedin.com/in/emadmokhtar/" target="_blank"><i class="fa fa-linkedin"></i></a></li>
<li><a class="sc-stack-overflow" href="http://stackoverflow.com/users/373051/emad-mokhtar" target="_blank"><i class="fa fa-stack-overflow"></i></a></li>
<li><a class="sc-facebook" href="https://www.facebook.com/emadmokhtarframework/" target="_blank"><i class="fa fa-facebook"></i></a></li>
<li><a class="sc-twitter" href="https://twitter.com/emadmokhtar" target="_blank"><i class="fa fa-twitter"></i></a></li>
</ul>
</div>
</aside>
<main>
<nav>
<a href="http://www.emadmokhtar.com">Home</a>
<a href="https://emadmokhtar.github.io/resume">Resume</a>
<a href="http://www.emadmokhtar.com/feeds/all.atom.xml">Atom</a>
<a href="http://www.emadmokhtar.com/feeds/all.rss.xml">RSS</a>
</nav>
<article>
<header>
<h1 id="how-to-run-celery-as-daemon">How to run Celery as daemon?</h1>
<p>Posted on Tue 20 September 2016 in <a href="http://www.emadmokhtar.com/category/django.html">Django</a> <b>Read in 3 min.</b></p>
</header>
<div>
<p><img alt="celery smoothy" src="http://www.emadmokhtar.com/images/celerysmoothie.jpg"></p>
<h1>Install upstart script</h1>
<h2>Permissions</h2>
<p>We need to make the user which will be used to run celery an owner of <code>log</code> and <code>run</code> directories.</p>
<div class="highlight"><pre><span></span>sudo chown -R root:root /var/log/celery/
sudo chown -R root:root /var/run/celery/
</pre></div>
<h2>Celery upstart</h2>
<p>1- Download <code>celeryd</code> file from <a href="http://github.com/celery/celery/tree/3.1/extra/generic-init.d/">Celery official github repository</a>.</p>
<p>2- Copy <code>celeryd</code> to <code>/etc/init.d/celeryd</code></p>
<div class="highlight"><pre><span></span>sudo cp celeryd /etc/init.d/
</pre></div>
<p>3- Make it executable</p>
<div class="highlight"><pre><span></span>sudo chmod +x celeryd
</pre></div>
<p>4- Make it run on startup</p>
<div class="highlight"><pre><span></span>sudo update-rc.d celeryd defaults
sudo update-rc.d celeryd <span class="nb">enable</span>
</pre></div>
<p>5- make it owned by root</p>
<div class="highlight"><pre><span></span>sudo chown root:root celeryd
</pre></div>
<p>6- Done, now you can start/stop/restart celeryd as service</p>
<div class="highlight"><pre><span></span>sudo service celeryd start
sudo service celeryd stop
sudo service celeryd restart
</pre></div>
<h2>Celerybeat upstart</h2>
<p>1- Download <code>celerybeat</code> file from <a href="http://github.com/celery/celery/tree/3.1/extra/generic-init.d/">Celery official github repository</a>.</p>
<p>2- Copy <code>celerybeat</code> to <code>/etc/init.d/celerybeat</code></p>
<div class="highlight"><pre><span></span>sudo cp celerybeat /etc/init.d/
</pre></div>
<p>3- Make it executable <code>sudo chmod +x celerybeat</code></p>
<p>4- Make it run on startup</p>
<div class="highlight"><pre><span></span>sudo update-rc.d celerybeat defaults
sudo update-rc.d celerybeat <span class="nb">enable</span>
</pre></div>
<p>5- make it owned by root</p>
<div class="highlight"><pre><span></span>sudo chown root:root celerybeat
</pre></div>
<p>6- Done, now you can start/stop/restart celerd as service</p>
<div class="highlight"><pre><span></span>sudo service celerybeat start
sudo service celerybeat stop
sudo service celerybeat restart
</pre></div>
<h1>Configuration file</h1>
<p>We'll configure celery to work with Django App, take care that you need to export <code>DANGJO_SETTINGS_MODULE</code> in order celery to discover your tasks. Configuration file need to be in <code>/etc/default/celeryd</code> folder.</p>
<div class="highlight"><pre><span></span>sudo nano /etc/default/celeryd
</pre></div>
<h2>Sample Configuration</h2>
<p>This configuration example is for a Django project name <strong>django_project</strong> located at <strong>/home/django_project</strong>, and we'll use same configuration file for <code>celeryd</code> and <code>celerybeat</code>.</p>
<div class="highlight"><pre><span></span><span class="c1"># Django settings module</span>
<span class="nb">export</span> <span class="nv">DJANGO_SETTINGS_MODULE</span><span class="o">=</span><span class="s1">'settings'</span>
<span class="c1"># Names of nodes to start</span>
<span class="c1"># most people will only start one node:</span>
<span class="nv">CELERYD_NODES</span><span class="o">=</span><span class="s2">"worker1"</span>
<span class="c1"># but you can also start multiple and configure settings</span>
<span class="c1"># for each in CELERYD_OPTS (see `celery multi --help` for examples):</span>
<span class="c1">#CELERYD_NODES="worker1 worker2 worker3"</span>
<span class="c1"># alternatively, you can specify the number of nodes to start:</span>
<span class="c1">#CELERYD_NODES=10</span>
<span class="c1"># Absolute or relative path to the 'celery' command:</span>
<span class="nv">CELERY_BIN</span><span class="o">=</span><span class="s2">"/usr/local/bin/celery"</span>
<span class="c1">#CELERY_BIN="/virtualenvs/def/bin/celery"</span>
<span class="c1"># App instance to use</span>
<span class="c1"># comment out this line if you don't use an app</span>
<span class="nv">CELERY_APP</span><span class="o">=</span><span class="s2">"django_project_name"</span>
<span class="c1"># or fully qualified:</span>
<span class="c1">#CELERY_APP="django_project_name.tasks:app"</span>
<span class="c1"># Where to chdir at start.</span>
<span class="nv">CELERYD_CHDIR</span><span class="o">=</span><span class="s2">"/home/django_project/"</span>
<span class="c1"># Extra command-line arguments to the worker</span>
<span class="nv">CELERYD_OPTS</span><span class="o">=</span><span class="s2">"--time-limit=300 --concurrency=8"</span>
<span class="c1"># %N will be replaced with the first part of the nodename.</span>
<span class="nv">CELERYD_LOG_FILE</span><span class="o">=</span><span class="s2">"/var/log/celery/%N.log"</span>
<span class="nv">CELERYD_PID_FILE</span><span class="o">=</span><span class="s2">"/var/run/celery/%N.pid"</span>
<span class="c1"># Workers should run as an unprivileged user.</span>
<span class="c1"># You need to create this user manually (or you can choose</span>
<span class="c1"># a user/group combination that already exists, e.g. nobody).</span>
<span class="nv">CELERYD_USER</span><span class="o">=</span><span class="s2">"celery"</span>
<span class="nv">CELERYD_GROUP</span><span class="o">=</span><span class="s2">"celery"</span>
<span class="c1"># If enabled pid and log directories will be created if missing,</span>
<span class="c1"># and owned by the userid/group configured.</span>
<span class="nv">CELERY_CREATE_DIRS</span><span class="o">=</span><span class="m">1</span>
<span class="c1"># CELERY BEAT</span>
<span class="nv">CELERYBEAT_CHDIR</span><span class="o">=</span><span class="s2">"/home/django_project/"</span>
<span class="nv">CELERYBEAT_OPTS</span><span class="o">=</span><span class="s2">"--schedule=/var/run/celery/celerybeat-schedule"</span>
</pre></div>
<blockquote>
<p>Note: <code>CELERYD_USER</code> & <code>CELERYD_GROUP</code> is the user will use to run celery tasks from Django app.</p>
</blockquote>
</div>
<div class="tag-cloud">
<p>
<a href="http://www.emadmokhtar.com/tag/django.html">django</a>
<a href="http://www.emadmokhtar.com/tag/python.html">python</a>
<a href="http://www.emadmokhtar.com/tag/celery.html">celery</a>
</p>
</div>
<div id="disqus_thread"></div>
<script type="text/javascript">
var disqus_shortname = 'emadmokhtarsframework';
(function() {
var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;
dsq.src = '//' + disqus_shortname + '.disqus.com/embed.js';
(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
})();
</script>
<noscript>Please enable JavaScript to view the <a href="https://disqus.com/?ref_noscript" rel="nofollow">comments powered by Disqus.</a></noscript>
</article>
<footer>
<p>© Emad Mokhtar </p>
<p>Built using <a href="http://getpelican.com" target="_blank">Pelican</a> - <a href="https://github.com/alexandrevicenzi/flex" target="_blank">Flex</a> theme by <a href="http://alexandrevicenzi.com" target="_blank">Alexandre Vicenzi</a></p> </footer>
</main>
<!-- Google Analytics -->
<script type="text/javascript">
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-11401860-2', 'auto');
ga('send', 'pageview');
</script>
<!-- End Google Analytics -->
<script type="application/ld+json">
{
"@context": "http://schema.org",
"@type": "BlogPosting",
"name": "How to run Celery as daemon?",
"headline": "How to run Celery as daemon?",
"datePublished": "2016-09-20 15:00:00+03:00",
"dateModified": "",
"author": {
"@type": "Person",
"name": "Emad Mokhtar",
"url": "http://www.emadmokhtar.com/author/emad-mokhtar.html"
},
"image": "{static}/images/celerysmoothie.jpg",
"url": "http://www.emadmokhtar.com/how-to-run-celery-as-daemon.html",
"description": "Install upstart script Permissions We need to make the user which will be used to run celery an owner of log and run directories. sudo chown -R root:root /var/log/celery/ sudo chown -R root:root /var/run/celery/ Celery upstart 1- Download celeryd file from Celery official github repository. 2- Copy celeryd to /etc/init.d/celeryd sudo cp celeryd /etc/init.d/ 3- Make it executable sudo chmod +x celeryd 4- Make it run on startup sudo update-rc.d celeryd defaults sudo update-rc.d celeryd enable 5- make it owned by root sudo chown root:root celeryd 6- Done, now you can start/stop/restart celeryd as service sudo service celeryd start sudo service celeryd stop sudo service celeryd restart Celerybeat upstart 1- Download celerybeat file from Celery official github repository. 2- Copy celerybeat to /etc/init.d/celerybeat sudo cp celerybeat /etc/init.d/ 3- Make it executable sudo chmod +x celerybeat 4- Make it run on startup sudo update-rc.d celerybeat defaults sudo update-rc.d celerybeat enable 5- make it owned by root sudo chown root:root celerybeat 6- Done, now you can start/stop/restart celerd as service sudo service celerybeat start sudo service …"
}
</script></body>
</html>