Scaling OwnCloud with Red Hat Storage

3. MariaDB multi-master clustering with Galera

So I don’t repeat myself from my previous article (found here), I will cover the Galera configuration process quite quickly.

Note: please be aware you will need to have the Galera yum repository available on all of your database servers.

 

Do the following on all database servers.

Configure Yum repository for Galera

Create a new file called /etc/yum.repos.d/galera.repo with the below contents.

[mariadb]
name = MariaDB
baseurl = http://yum.mariadb.org/5.5/rhel6-amd64
gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB
gpgcheck=1

Install required packages

Install the necessary packages to get MariaDB and Galera ready to be used. To do this, use yum to install the below packages.

yum install -y MariaDB-Galera-server MariaDB-client

Configure MariaDB

We need to enable our local installation to allow access from other servers in order to have Galera set up properly. To do this, do the following.

service mysql start
mysql -e "GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'password' WITH GRANT OPTION;"
mysql -e "GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' IDENTIFIED BY 'password' WITH GRANT OPTION;"
service mysql stop

Now edit your local /etc/my.cnf file on each database server so it appears as follows

#
# This group is read both both by the client and the server
# use it for options that affect everything
#
[client-server]

#
# include all files from the config directory
#
!includedir /etc/my.cnf.d

[mariadb]
wsrep_cluster_address=gcomm://galera01.example.com
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_sst_auth=root:password
wsrep_sst_method=rsync
log-error=/var/log/mysql.log

 Note: You will notice that the above config points all systems to “galera01.example.com” as the server to join the cluster. You cannot have “galera01.example.com” pointing to itself, so on this server, you need to point this to another server. 

Firewall

If you leave iptables filtering your systems, don’t forget to open the necessary ports for Galera to function.

To do this, run the following.

for x in 3306 4444 4567 4568 ; do iptables -I INPUT -p tcp --dport $x -j ACCEPT ; done

Lastly, we want our Galera database cluster to work behind the VIP (10.0.1.20) of our Load Balancer. Because we are using a Direct Routing method, we need to tell iptables to remap our database connections to that IP. You can do this with the following.

iptables -t nat -A PREROUTING -p tcp -d 10.0.1.20 --dport 3306 -j REDIRECT
service iptables save

Enables Services

Here you can do you’re regular thing to ensure everything is running and set to load on reboot. To do this, run the following.

chkconfig mysql on

Starting the cluster

As we have all the systems pointing to “galera01.example.com” to join the cluster, it is important to ensure that the cluster is already available before starting all other servers.

To start the cluster from “galera01.example.com” when the entire cluster is in an offline state, run the following.

service mysql start --wsrep-cluster-address="gcomm://"

On all other servers, you can perform a normal service start once the primary server is online.

service mysql start

 

Provided you have not received any error messages, your cluster should be completely online. You can verify this by creating a temp database on one system and it will appear on all other nodes if everything is working as expected.

If you jump across to your master load balancer, you can also see that our 3 MariaDB servers have now popped up in the list of load balancing servers.

[root@lb01 ~]# ipvsadm -l
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
 -> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP galera.example.com:mysql rr
 -> galera01.example.com:mysql Route 100 0 0 
 -> galera02.example.com:mysql Route 100 0 0 
 -> galera03.example.com:mysql Route 100 0 0 
TCP owncloud.example.com:http rr persistent 30
TCP owncloud.example.com:https rr persistent 30
[root@lb01 ~]#

12 comments on “Scaling OwnCloud with Red Hat Storage

  1. Jan Dam November 26, 2013 14:39

    Thanks for this excellent article!

  2. Patrick November 27, 2013 17:19

    Dale: thanks for this great article. The BZ you mentioned got “CLOSED NEXTRELEASE ” ages ago which suggests that the fix could be part of the latest selinux-policy by now. Yet no “Fixed in version” is mentioned while Miroslav usually adds the selinux-policy release in which it is fixed. Puzzling.

  3. Patrick November 27, 2013 17:27

    It seems the selinux-policy in RHEL6.4 has a fix:
    $ getsebool -a | grep httpd_use_fusefs
    httpd_use_fusefs –> off

    So it should be just a matter of:
    $ sudo setsebool -P httpd_use_fusefs on
    to give Apache the ability to use GlusterFS storage.

    • Dale Macartney November 27, 2013 21:41

      Thanks Patrick, I was hoping for some good news like that.

      I’ve just updated the article to reflect the changes.

  4. tquang April 24, 2014 19:12

    After configurated KeepAlived and started it, I no see listen port (3306) on both servers:

    [root@lb1 ~]# netstat -natp
    Active Internet connections (servers and established)
    Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
    tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1033/sshd
    tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1109/master
    tcp 0 0 192.168.56.103:22 192.168.56.1:54793 ESTABLISHED 1201/sshd
    tcp 0 0 :::22 :::* LISTEN 1033/sshd
    tcp 0 0 ::1:25 :::* LISTEN 1109/master

  5. hoangvu August 8, 2014 18:15

    Thank for article!
    But in my system build, owncloud website is very slow.
    I find that gluster processes in rhs and web servers use high CPU when
    load website (about 75-90%).

    • hoangvu August 8, 2014 18:42

      Oh, after use NFS instead Gluster to mount Gluster volum in web servers, everything is ok!

  6. luli June 2, 2015 13:04

    Create tutorial,
    l am trying to implement this topology, but l am facing a problem, and also have one question
    should l need to install owncloud in all 3 server ( apache web farm ), so can then replicate eachothers ?

  7. theluli August 2, 2015 15:01

    Hi Dale
    You got nice tutorial,
    l tried your things but l am facing one problem , since l am using only to apache web server with owncloud , should owncloud be installed in 2 servers ? .
    Even that l have install , am unable to work with 2 server is same time as fail-over , or load balancing
    Can you please advise me on this matter

  8. Stéphane January 20, 2016 09:00

    Great, great article Dale !
    My system works fine …

  9. theluli February 13, 2016 14:17

    Very nice article , but need so much servers
    Since l am just training for this kind of things, l have one question
    My setup is with 2 load-balancer with haproxy , and 2 web-server , setup if working very fine , but how l can get of use https instead of http

    Please help me in this matter
    Thanks again for nice tutorial

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>