First off, I work for a college. I need to use CRD to send emails to students who are enrolled for Direct Deposit Check (these are personalized emails containing their check amounts, deductions, etc). I need to send around 200,000 - 300,000 emails weekly to these students.
I have been doing some testings with 89 emails, and I see that it takes around 30 minutes or more to finish sending the emails - I'm embedding the report (which contain student's info) into the email as TEXT. The email is not that long.
I wanted to know if CRD can handle sending all the emails, and if there is a way to speed up the process.
Yes, we bang out 6000 reports every weekend, so you shouldn't have any problems. The things that affect the speed are:
- How fast your PC processor is and how much memory you have available
- Whether you have set processing in the reports to happen on the server or locally
- How fast your report itself takes to generate
- How fast your email server is. CRD sends the email one at a time, and checks to make sure that have gone or failed before it moves on to the next report.
I don't think embedding would decrease the speed vrs attaching. Of course, if you were exporting to folder that would be a lot faster that any emailing method. and based on what Tina said in your earlier post, I don't think that html or text format makes a noticeable difference.
You can further speed up the process by checking the "multi-threading" check box in Options - General.
Unfortunately, CRD is limited by the capabilities of the CR runtime components. They only allow up to 4 reports to be run simultaneously, so thats the maximum number of threads that CRD will be able to run at any given time. Still, that's 4 times more than before, so your current 89 reports should take a significantly shorter time to process.
Finally, and this is how we do it, you may want to look at the Corporate Edition. We have set it up as follows:
There are 5 standard desktop PCs. One is the master and the other 4 are slaves. Each slave has multi-threading switched on. The master controls the scheduling database and tells each slave what to run and when. So that gives us up to 16 reports running simultaneously.
I have noticed, thought, that where I was using dynamic schedules, I have had to split the dynamic schedule in "smaller" dynamics. e.g each schedule only handles a letter of the alphabet. This is because CRD sends an instruction to a slave saying "run the dynamic schedule". The slave can handle 4 reports simultaneously, but CRD's instructions are to schedules, not reports. So by splitting it up, I have managed to get say a, b,c and d's running on one box, e's f's g's and h's on another and so on.
Our reports are quite beefy too. Each report takes about 55 seconds to generate so it still takes a while. But its a lot less than it used to be.
A friend of mine in another company didn't go for the Corporate Edition. She just installed 6 CRD's on 6 separate pcs. But then she has to maintain the schedules on 6 machines which she finds is a pain.
I tried "Multi-threading" and it did speed up the process by 5 minutes. However, I tried running it from my co-worker's machine that has 2GB of memory (vs mine that has 256KB) and it took the same amount of time. We have the same processor speed.
I'm also processing the reports locally, but I'll try putting it on a server to test. The report itself generates fast. About the email server, I'm not sure about that one.
How long does your 6000 reports take to complete? Remember, I have 200,000 - 300,000 emails to send. Thanks for describing your set up.
The Corporate Edition seems like the only way to go then. For that, I'll have to consult it with my boss.
My report is not embedding well, so I may not use this afterall.
Takes about 6 to 8 hours on a Saturday. We do it then because thats when the database server and the mail server are quietest.
Its more the processor capablitity of the PC than the memory. Don't forget, running 4 reports simultaneously will only run 4 times as fast if your processor can also work 4 times as fast. Otherwise you are simply sharing the same processor power accross 4 processes and so you will not see huge improvement. Each process will, in fact, take slightly longer. And, of course, there is the speed of your mail server.
How much time have you got to send 300000 emails? Hours, days? If I were doing that many and buying the Corporate, I think I might ask the CRD people to do some bespoke stuff for me and remove some of the email checks etc which slow down email.
I remember that they speeded up the "round robin" process for me (where the master tells which slave what report to run and when) when I asked, and gave me some pretty good tips on how everything hung together and worked.
One of the tips was that CRD was deliberately slowed down on the email delivery front so that it would discourage nasty people from using it as a spamming tool. When they were fairly certain that I was not a spammer, they sped up this part for me. (Unfortunately, as my email server is fairly slow at the best of times, it only made a small difference).
Anyways, good luck with it, and keep us posted as to how you get on.
I did a run test yesterday with 1210 emails and it completed in 3 hours. However, 200,000-300,000 emails would take forever.
We are trying to see what we can do, or maybe use another software that's targeted to sending emails.
I'm still having problems embedding the report because the HTML gets all jumbled up, so if I cannot solve that problem, then there would be no point in sending the emails.
