Resource timeout.. please try again

[mcavalcax]

I was due an update on my connector as MT4 stopped working with the old bridge version, I went to Github and got the new v2.0.1 mql4(last mth update) and python(2mths old) snippets and updated on my computer, I am using Python 3.6 on Spyder and MT4 Build 1220 (Sept2019), I also update libsodium and mql and zmq libraries in include and library just in case

I managed to establish the connection, however, the rate of updating of the Dictionary _Market_Data_DB went down considerably, whereas the expert tab on MT4 carries on bursting PUB Socket updates (1 update per second per ccy), the rate of build up of the dictionary on the Python side is much slower (for e.g. GBPUSD and GBPJPY get on 2 updates per minute ,AUDUSD 1 update per minute in the London mng? Before was one update every 1-3 secs)

I see there are some new features on the Python side a pool_timeout and sleep_delay, I don’t want to mess with it and brake it even more, if I can get some advice on what settings I can use to speed up the Market Data updates would be a great help as I use the spot from MT4 from Market_Data_DB as an input for the model and 2 quotes per minute is clearly not enough

Thanks in Advance!

Marcello

[mcavalcax]

Btw EURUSD carries on updating every second on ._Market_Data_DB, it is the others that are an issue and only update once or twice per minute, really puzzling

[mcavalcax]

Manage to get it to work, behaviour on MT4 side has changed. Before MT4 would stream constantly regardless of what was happening on the Python side, now, once you re-start your Python connector you DO need to re-start the MQL4 connector as well by simply going to properties and hitting the OK button, that will re-start the connector afresh from the MT4 side

[mcavalcax]

Gents, while the connection is working there is a new issue that was not happening before, I just missed two trades (on the demo account, no sweat) and got the mesage : Resource timeout.. please try again
Both CPU and Memory running below 50%, both the spyder/python and MT4 running on a Intel I7 3.6GHz Processor with 40GB of RAM, it is certainly not the computer resources, any light here pls?

[mcavalcax]

A bit more information on the issue, regarding dependencies:
the dependencies described on the GitHub project https://github.com/darwinex/dwx-zeromq-connector :
• Python: (minimum v3.6) – I am at 3.6.8
• libzmq: (minimum v4.2.5) – the one available on the repository is v 4.2.1, which seems to be the same as before the last update but just highlighting
• pyzmq: (minimum v17.1.2) – I use 18.0.0

regarding the behaviour of the connector:
Once both MT4 and Python connectors have been initialised and I subscribe for Data via Python Connector, the connector works well for trading instructions i.e. I can open and close as many trades as I want but the subscription doesn't work well, i.e PUSH Port is working but SUB Port not working properly

For e.g. I subscribe for EURUSD, AUDUSD, GBPUSD, GBPJPY, USDJPY, if I go to the dictionary and look at ._Market_Data_DB only EURUSD is really working the others get one update per minute if any, below you can see on the timestamps eurusd is working, all the other ones are not, despite the Expert tab on MT4 be bursting with update on all 5 currencies, GBPJPY didnt get any updates...

'2019-10-16 19:33:27.022809': (1.10757, 1.10761),
'2019-10-16 19:33:27.134544': (1.10758, 1.10761),
'2019-10-16 19:33:29.835253': (1.10758, 1.1076),
'2019-10-16 19:33:30.038707': (1.10758, 1.10759)},
'AUDUSD': {'2019-10-16 19:30:11.895641': (0.67632, 0.67637),
'2019-10-16 19:32:28.069566': (0.67626, 0.67632)},
'GBPUSD': {'2019-10-16 19:32:01.289154': (1.2833, 1.28343)},
'USDJPY': {'2019-10-16 19:32:14.814297': (108.727, 108.732)}}

If I then go to MT4 on the graph where the EA is and ask for properties of the EA, just double click on any variable without changing them and just hitting Ok the EA re-initialises, and then the subscription starts bursting Data on the python side on all ccys subscribed but the trading functionality stops working, so the SUB Port starts working but the PUSH Port brakes down

Any help on this issue is appreciated
Thanks
Marcello

[mcavalcax]

I tried to change the order of the ccys on MQL4 string Publish_Symbols and learned that only the first ccy gets constantly updated on Python ._Market_Data_DB the other ccys just get rarely few updates, I hope this information helps as I am not familiar with the inner guts of the connector. Thx

[integracore2]

Hi @mcavalcax ,

Thank you for providing an extensive account of observations!

We're going to try and reproduce the issue, then come back to you with a solution - stay tuned.

[integracore2]

@mcavalcax - just a quick note in addition to the previous message. Please bear in mind that the symbol you deploy the EA to will affect the frequency of updates as well.

This is because the code that fires off bid/ask updates is enclosed inside the OnTick() method in the EA, an event that fires when a new tick arrives on the chart the EA is deployed on.

[mcavalcax]

Agreed that depending on the currency the frequency of the updates will be different, but as described above the first ccy listed on the mq4 EA under the Publish Symbols string gets most of the updates whereas the other ones dont get any. If I put AUDUSD first and EURUSD second, AUDUSD will get roughtly 1 update per second and EURUSD 1 update per minute if any, when EURUSD comes first it gets between 1 and 3 updates per second

[integracore2]

Happy to inform you that we've isolated the issue and are working on a patch 🙂

Stay tuned!

Fix output below:

[mcavalcax]

Fantastic News, I am happy to run heavy testing on it once the patch is in. From: Ali Saif <notifications@github.com> Sent: 06 November 2019 14:25 To: darwinex/dwx-zeromq-connector <dwx-zeromq-connector@noreply.github.com> Cc: mcavalcax <mcavalca@hotmail.com>; Mention <mention@noreply.github.com> Subject: Re: [darwinex/dwx-zeromq-connector] Resource timeout.. please try again (#48) Happy to inform you that we've isolated the issue and are working on a patch 🙂 Stay tuned! [DWX_ZeroMQ_Market_Subscription_Fix_1]<https://user-images.githubusercontent.com/33300041/68306318-9ce5cb80-00a9-11ea-9c69-4b77e0eef74a.gif> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#48?email_source=notifications&email_token=AG675GH45IPGKOS467DP2L3QSLHUTA5CNFSM4JAZZB4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDGWOCY#issuecomment-550332171>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AG675GFACE5A2ZNHWBITXLDQSLHUTANCNFSM4JAZZB4A>.

[teddywaweru]

Hi Everyone.
On the matter of resource timeout when capturing Historical data for multiple instruments to _History_DB, I found that including a time.sleep(0.005) counter after carrying out each request solved it. I think it's a matter of too many requests to the same port, as well as re-writing the same dict multiple times. Gratefully, the execution speed isn't greatly affected.