MySQL Ndb & MySQL with Galera why we should not compare them.
Written by Marco Tusa
In the last few months, we have seen more and more discussion on the use of Galera solution for MySQL Clustering.
I have being one of those that had heavily test and implement Galera solution, actually with quite good results and I have also presented SOME of them at Oracle Connect.
On the other side I have be work with MySQL NDB for years (at least from 2007) at many customers site, from simple to complex setups.
So also if I cannot consider myself as mega expert, I think to have some good experience and insight on both platform.
The point here is that I was not happy in reading some articles comparing the two, not because the kind of tests, or results.
Not because I prefer this or that, but simply because, from my point of view, it does not make any sense to compare the two.
We can spend pages and pages in discussing the point here, but I want try to give a simple generalize idea of WHY it makes no sense in few lines.
NDB brief list
NDB is not a simple storage engine and can work independently, MySQL is “just “ a client.
NDB is mainly an in memory database and also if it support table on disk the cost of them not always make sense.
NDB is fully synchronous, no action can be returned to client until transactions on all nodes are really accepted.
NDB use horizontal partition to equally distribute data cross node, but none of them has the whole dataset (unless you use one node group only, which happens ONLY when you don’t know how to use it).
NDB Replicate data for a specific factor, which is the number of replica, and that replication factor will not change with the increase of the nodes number.
Clients retrieve data from NDB as whole, but internally data is retrieve by node, often use parallel execution. (Not going in the details here on the difference between select methods like match by key, range, IN option and so on).
NDB scale by node group that means it really scale in the Dataset dimension it can manage and operation it could execute, and it really scale!
Galera brief list
Galera is an additional layer working inside the MySQL context.
Galera require InnoDB to work.
Galera offer “virtually synchronous” replication.
Galera replicate the full dataset across ALL nodes.
Galera data replication overhead, increase with the number of nodes present in the cluster.
Galera replicate data from one node to cluster on the commit, but apply them on each node by a FIFO queue (multi thread).
Galera do not offers any parallelism between the nodes when retrieving the data; clients rely on the single node they access.
So why they cannot be compare?
It should be quite clear that the two, are very different from starting from the main conceptualization, given NDB is a cluster of many node groups with distribute dataset, while Galera is a very efficient (highly efficient) replication layer.
But just to avoid confusion:
NDB does data partition and data distribution with redundant factor.
Galera just replicate data all over.
NDB apply parallel execution to the incoming request, involving more node groups in data fetch.
Galera is not involved at all in the data fetch and clients need to connect to one node or more by themselves, means application require managing parallel request in case of need.
In NDB the more node groups you add the more you get in possible operation per second and data archived/retrieved.
In Galera the more nodes you add, the more overhead you generate in the replication, so more data will require to be “locally” commit by the replication layer, until when the number of nodes and operations executed on them will compromise the performance for each node.
NDB Cluster is a real cluster solution, design to internally scale and to perform internally all the required operation to guarantee high availability and synchronous data distribution.
Galera is a very efficient solution to bypass the current inefficient mechanism MySQL has for the replication.
Galera allow to create a cluster of MySQL nodes, in virtually synchronous replication. This with almost zero complexity added on the standard MySQL management.
Never the less the obtaining platform is composed by separate nodes, which for the good or the bad is not a system of distributed data.
Given that, the scenario where we can use Galera or NDB are dramatically different, trying to compare them is like comparing a surfboard with a snowboard.
I love them both, and honestly I am expecting to see Galera deployment to dramatically increase in 2013, but I am still respecting my motto “use the right tool for the job”.
Let us try to make our life easier and avoid confusions.
On the 3td of July we have done our first year in Canada, our first anniversary, an important milestone. Thinking back to 1 years ago or even more to 2 years ago when I join the company, it's sounds very surprising how many difficult goals we had successfully achieve. That because when moving a whole family, the number of elements that must be organized and keep on the right track are much more then the one moving alone or just in two people.
The merit of this success doesn't reside on me only, but on all my family that have being working as a team, all the time dealing with the issues we were facing all together.
In short another success story of teamwork!
Why moving to Canada
As reward for the unbelievable effort our kids did during the past year, we decided to send them to Italy for the vacation. We know they would have loved to spend their time at the beach and sea with cousins and friends.
When they come back my wife and I had a discussion with my son. He asks us to explain him why we decide to move to Canada, and why we decide to remain here.
The question raised after one year was not so silly, and was a good one giving he was not including in it simple topics that could drive to wrong path, like because we like or dislike this or that.
The question was WHY the real long term reason why we choose to move.
Coming from some really underdeveloped country will provide an easy answer, but coming from Europe will make the whole story more difficult.
Am I sure I want to move?
We did decide to move before I start to work in Pythian, at that time we were evaluating few options keeping into account the destination country and the job offer.
So yes, we did our consideration about it, considering the effort/cost of getting what we want in our country, and what it will be in the country we choose. In that respect we have done a comparison research using mainly the indicators coming from the UBS bank report, defining a short list of possible countries.
To be honest Canada was not in the top three. So why we choose it at the end?
Because the long-term view was indicating that, the country and the company was the right choice. It was the long-term plans that make the difference, when moving a family, the short term should be consider as 3-4 years, but long term is 10 years or more correctly 20 years. Having the chance to get that prospective inside a company is not an easy finding. Finding a country that could host that is not easy as well, so that's why Canada.
Am I moving with a job?
When you have a job already, also if it is not the perfect one, then make no sense to move without it. If you don't have a job then don't move only because a job offer, it will not gives you a better life in long term, unless you have other reasons for the move.
Common sense apply, in my case I had a good job and good position, so will not have move if the alternative would not have be more then interesting.
What about the family?
I cannot move without my family, and actually we choose to move mainly because the kids, so no sense to left them behind. More a family is like a small company base on teamwork, it must act in sync and cannot or should not be split.
What to do before moving
Once decided where to go and having the company sponsoring us, what we had to do was, asking for the work permit, find a decent house, find the school for the kids, all not in this order.
We decided that our priority would have been: School for the kids, House, Work permit, this to remain consistent to our main idea of why moving.
Schools in Ottawa can be Catholic or public, if you are not catholic well easy ... but if you are, that add a variable. Then them can be English or French, given our kids did not talk English (or French) we decide that at least for the first 2 years, Catholic English would have be the best choice.
At that point the issue was to find a school with a good rate, to do that we used the Ottawa site. A good rate is 8, careful to check the history, better to have a 8.1 moving from 7 then a 8.8 moving from 9. We choose the St Marguerite d'Youville in Greenboro, not the best, but the institute is trying to do better. We also spoke with the principal and teachers we like them, oh yes you must plan at least a visit for the house before moving, and the schools, so what we did was to have a list of 3 schools and then when in Ottawa, fix a meeting with the school to get direct feedback.
After that choosing an home is just a matter of time, the ratio to choose the house is 5 minute walking from the school, then 20 minutes from work, and 10 minutes max from the closest food market.
Crossing the line and assign a value of each variable, a value from 1 to 10 in relation to the proximity, helps us to identify where to focus for the house search.
To do the practical search we used this site also we found a good agent (Rocco Manfredi) that was helping us a lot, being really honest and direct in providing advices.
When we had all these in line, we go ahead with the work permit, obviously having the company sponsoring you is good, it helps you a lot. In my case I had apply long ago for working visa in Canada, and I was already approve, at that time I did not move given my work in other countries, but I was quite confident we would not have issue also this time.
When apply for the first time, you can choose to ask for one year work permit, as well as more, given the minimum for applying for permanent work visa is two years once in Canada, why apply for less then that? I ask for three years and I got it in less then a month. That gives us the time to apply directly for the permanent without bothering human resources with additional work, and without putting us under stress. So longer is better, suggestion is to ask for it.
When is the right time to move?
We came from a warm country, so we decide to move in the right season to get use to the different climate, but we also have to wait for the school to be over. So we decide for the 3 of July (immediately after the Canada Day), one week after the school is over in Italy.
My personal suggestion is never, ever come here in winter or in September/October, you will not have the time to adjust and will suffer more then what you will do in any case.
Also Ottawa is amazing in the summer, and there are so many things to do here that it will help a lot to tune your body and mind.
Having the chance to buy the right cloth for the winter takes a little bit of investigation and understanding.
Coming from a place where -1 Celsius is a crazy cold, we were not prepared to manage -40 C, or at least that require some additional understanding. Coming here in the right season give us the chance to talk with neighbors, friends and colleague to collect advice and do the right choice when go shopping.
What to do once here?
Here I give few general advices on the base of my direct experience.
Canada is a country with high level of immigrants, and a limited residential population, comparing it with the available space and resource.
More Canada needs immigrants to have his economy prosper and to have people working and paying taxes. That is a very important point, because if you come here and you are willing to work, willing to participate to the country grow, then you will have all the possible help from the institutions. In short being an immigrant, specially if skilled, is not making you a B series, and this can be also seen from the huge effort, Canada institutions are doing to help immigrants to set up correctly.
What you should do once here is to follow little but very important point that will make you a productive resident, and at the same that secure you in case of issue like illness.
One good root resource is: http://ottawa.ca/en/social_com/immigration/index.html for general overview or http://www.ontarioimmigration.ca/en/after/index.htm .
Once you are in your room of the hotel, or pension of your day one, be ready to cover those things right away:
Apply for SIN
Open a bank account
Get information about working in Ontario
Find the services you need, close to home
Locate a doctor, dentist or other health services
Find a public library and other community services
Apply for the Canada Child Tax Benefit (CCTB)
Get a map of your community and learn about public transportation
Find language classes for you and your family
I did the SIN as soon as I arrive, this is a must to have a job, you can look here to find the one closest to your place. Do not forget, this is priority one, do it right away, as I did.
Then same day run and go for the HOIP, look here for the office closest to your place and look for a place that has "health" option.
Keep in mind that the Ontario will not cover you right away; you must wait for three months before accessing the medical coverage with OHIP, for more information look here.
You think you have done? No way! Open a bank account if you did not already.
What I did is to open it BEFORE arriving, that for two reasons:
That allow you to transfer any money here before arrive;
Open it also if with few dollars in, give you access to the history credit system. The sooner you start to establish your history the sooner you will get good conditions from banks and insurance companies. So do not wait!
Choose the bank you like, I am with TD, so far so good.
One note, I am Italian and I am use to bargain on everything, Canadian seems not bargain or at least they state so, but that is not true, they do in a different way.
When you need to open an account in a bank or to find insurance or whatever, do no stop in the first place. Instead go around ask for condition and prices, and then clearly tell them that you are looking around. When you have the best condition/price do the tour again, and ask for better condition/price base on the latest good one. You will be really surprise of the results, I have seen my request scaling up manager to manager up to directors and so on, getting at the end VERY different numbers and conditions.
So you have your account, your SIN, your OHIP, what about the job? I assume you have it actually, so otherwise why are you here? In the case you don't have it this is a decent reference for it here and the job site is here and here.
Important note is that you have to cover all the above in the first week, or better two days after, from your arrive in Canada. Don't waste your time and do what you have to, FAST!
Discovering the neighborhood is important, so look around for the essential service, like pharmacies, clinics, food stores, and whatever you think relevant for your daily life.
About doctors, is important you start to look for a doctor as soon as possible, I did the mistake to wait and it takes me more then 8 month to have a family doctor, this because not all doctors takes new patients.
There is a site that gives you indication, but honestly it is useless.
Go check directly in the clinics close to you and see if they have spots, it is faster and makes much more sense then waiting for the system to call you.
Hey, you are going to pay taxes right? So start to use your own money! The community centers are there to support you in many activities, sport, arts, reading and so on. Use them, they are much cheaper then other service, or fully free (pay from the taxes). Quality is not number one? Who cares good enough for you to start, and remember you have already paid for it.
This site give you full indication of where the centers are, go there and register, it will be a very nice surprise for you to discover how many things you can do with your own money.
If you have kids and you are eligible you can apply to CCTB, look at this site for more information and to discover if you can apply.
Finally in Ottawa there are many languages spoken, but the official ones are English and French.
English is a must, and you must be able to communicate using it, but also French is very often request.
If you have wife or kids, it could make sense to help them in learning the languages, there are different programs that could help you, some private and very expensive, some from the government that are really affordable. Information at this site for a locator tool.
I think this is almost all about the basics, only missed things are the obvious one which is transportations, distance here are huge, so you cannot walk everywhere unless you have the whole day free. Using buses is a good way to move, just note that the monthly pass is not cheap, so unless you plan to use it EVERY day, it doesn't make sense.
Car are cheap as well, I mean by a basic one, what is not cheap is the insurance, specially the first year. But if you have kids and you want to live the city and the place around it, having a car is a must. Remember you must take the Ontario Driving license, yours is not valid here, and the examinants are really ... fussy. Web site for information here.
Keep also in mind that bicycle are also a good alternative, and almost everywhere you will see special path for them, so make sense to use them given cycling is good for health good for traffic and pollution.
Ok that is really all.
What to do to survive the winter
Let us go back to my direct experience. First thing we did, was decide that we will not be stop by the cold. Second I don't want to die under the snow so we pay for the snowplowing service.
Then, I love snow, and I like skating, skiing so for me no problem, I have a lot to do, different story for the rest of the family, but I did try to keep them with me, and at the end it worked out quite well.
At the end of the winter, we all were able to go skating around, and I can go skiing with my son that was enjoying it a lot.
Do not miss the skating on the canal or the winterlude festival, is a very god way to learn a different approach to the crazy cold it will invest you.
Winter is long, and I was aware I was not going to notice it also because the work, my kids will have to go to school so no issue as well, but what about my wife? She come here without a job, which could be nice for some time, but then it could become really ... boring, depressing and frustrating.
We decide, actually she decide, to look for something she could like, she start to look around and she found a job she likes in less then a week. Right now she goes to work every day at 6AM, which for me is crazy, but for her is ok because she loves it. Winter or summer no difference, this is important, really important.
SO advice is if you come here with your wife (or husband), help him to find a job, spending the day at home waiting for you is not only stupid, is dangerous.
Another point is, people don't go out when is cold, if you want to have friends you must call them and see them in the houses or pubs, if you organize the better.
This is a difference we have from much European country, where you have people gathering in the street, then moving somewhere despite the weather, and sometime just jumping inside your house uninvited.
After one year what is the balance?
Is not possible to do a balance after a year only. What I can say is that, so far so good, not yet fully able to understand the country or the people, but I feel more at home here then in many other place I had live in the past. It was fun for me to see during the Olympic Games my family cheer for Canada as they were doing for Italy, was a very small signal but really meaningful of how the Canadian spirit can take you.
My advice is, be flexible, adapt and DO NOT try to live your country here. Adapt the good from your country to your current life but then be open and adapt to what is the Canadian lifestyle.