Sunday, March 18, 2012

Installfest: Oracle 11g XE on Amazon Linux Micro Instance

This is just a quick (as in "I really, really mean it quick") note on installing Oracle 11g XE on Amazon EC2 on a vanilla Amazon Linux AMI as a micro instance. Cloudy...

Why Amazon Linux? It's easy and it's pre configured with most of what you need. Both cloud wise and for other things. It also gets good reviews as a production OS.

No, Oracle 11g XE on Amazon Linux this is not supported by Oracle. But then again, you will receive no official support on Oracle 11g XE by Oracle on any OS...

I have not validated if every step described below is necessary, by I reasoned that the standard install steps will stand, even if they upgrade the default Amazon Linux AMI.

If you have no experience with Linux or Amazon EC2, I think you will be better off with a more verbose installation guide than this is!

Create the Instance
If I am unclear (or just a tad abbreviated), just follow this documentation at any time.
  • Launch new instance, and choose "Basic Amazon Linux AMI YYYY.MM"
  • Choose instance type micro, set or create key pairs and set or create security group (everything else, just keep default)
Your are done creating the instance.

Connect to the Instance
I use PuTTY, for alternatives or a more elaborate description of how to connect with PuTTY, see this documentation. I assume you have PuTTY suite (with PuTTYGen) installed.

You also have to check that port 22 is open for the security group associated with the image.
  • Start PuTTYGen
  • Load the .pem file created from then key pair in use with the instance (switch to "all files" to see it)
  • Click Save Private Key, and Yes
  • Exit PuTTYGen
  • Start PuTTY
  • Copy Public DNS from EC2 Console
  • Paste Public DNS into Host Name in PuTTY Configuration
  • On left side, go to Connection -> SSH -> Auth and browse to the location of the previously generated ppk-file
  • Back to Session, give it a name  in "Saved Sessions" and Save
  • Click Open and Yes in the dialog
  • Connect as ec2-user
You are done connecting to your new instance.

Configure EC2 Tools
This you can do on your host machine, or the newly created instance. I will describe how for the newly created instance (less work for me, but also less secure for you!).
  • Log in as ec2-user
Configure root user:
  • sudo su - root
  • passwd <new password>
  • exit
  • su - root
  • <new password>
  • id
EC2 Tools are installed by default, but you have to let it know which credentials to use. Copy the pk and cert files into /root directory, and modify .bash_profile:
  • su - root
  • vi ./.bash_profile
  • export EC2_PRIVATE_KEY=/root/pk-<your pk>.pem
  • export EC2_CERT=/root/cert-<your pem>.pem
  • export EC2_URL=https://ec2.<region>.amazonaws.com
I use eu-west-1 as region. You should now be able to issue
  • ec2-describe-availability-zones
And receive a list of zones.

Configure Instance Storage
First, create and attach a new volume (30Gb) to the instance:
  • su - root 
  • ec2-describe-instances 
  • Take a not of the instance id and availability zone 
  • ec2-create-volume --size 30 --availability-zone <zoneSameAsTheInstance>
  • Take a note of the volume created
  • ec2-attach-volume <yourVolumeId> -i <yourInstanceId> -d <yourDeviceName>
  • Like this: ec2-attach-volume vol-xxxxxx -i i-xxxxxx -d /dev/sdf
  • ec2-describe-volumes
Format volume and make it stick on reboot:
  • mkfs -t ext3 /dev/sdf
  • tune2fs -m0 /dev/sdf
  • mkdir /u01
  • mount /dev/sdf /u01
  • vi /etc/fstab
  • Add the following line:
  • /dev/sdf    /u01        auto    defaults,noatime 0 0
Increase swap file (using the root device):
  • dd if=/dev/zero of=/home/swapfile bs=1024 count=2097152
  • mkswap /home/swapfile
  • vi /etc/fstab
  • Add the following line:
  • /home/swapfile swap swap defaults 0 0
This is a bit of a cheat, because the default swap disk will still attach, you cat alter that later.

Set host name for the server:
  • vi /etc/sysconfig/network
  • HOSTNAME=<yourHostName>
  • :wq
  • vi /etc/hosts
  • Add  <yourHostName> to the line containing 127.0.0.1
  • :wq
Restart to check if everything is OK:
  • shutdown -r now
At this stage I created a new AMI based on the current configuration (with the new 30Gb volume still attached). You never know :-)

Configure Instance for Oracle 11g XE
Since we can't use yum oracle-validated on the Amazon Linux, we will have to do this the hard way.

Create oracle user++:
  • su - root
  • groupadd oinstall
  • groupadd dba
  • useradd -g oinstall -G dba,oinstall oracle
  • passwd oracle
  • chown oracle:oinstall /u01
Get the latest packages:
  • yum install binutils-2.*
  • yum install compat-libstdc++-33*
  • yum install elfutils-libelf-0.*
  • yum install elfutils-libelf-devel-*
  • yum install gcc-4.*
  • yum install gcc-c++-4.*
  • yum install glibc-2.*
  • yum install glibc-common-2.*
  • yum install glibc-devel-2.*
  • yum install glibc-headers-2.*
  • yum install ksh-2*
  • yum install libaio-0.*
  • yum install libaio-devel-0.*
  • yum install libgcc-4.*
  • yum install libstdc++-4.*
  • yum install libstdc++-devel-4.*
  • yum install make-3.*
  • yum install sysstat-7.*
  • yum install unixODBC-2.*
  • yum install unixODBC-devel-2.*
Check the parameter settings:
  • /sbin/sysctl -a | grep fs.suid_dumpable
  • /sbin/sysctl -a | grep fs.aio-max-nr
  • /sbin/sysctl -a | grep fs.file-max
  • /sbin/sysctl -a | grep kernel.shmall
  • /sbin/sysctl -a | grep kernel.shmmax
  • /sbin/sysctl -a | grep kernel.shmmni
  • /sbin/sysctl -a | grep kernel.sem
  • /sbin/sysctl -a | grep net.ipv4.ip_local_port_range
  • /sbin/sysctl -a | grep net.core.rmem_default
  • /sbin/sysctl -a | grep net.core.rmem_max
  • /sbin/sysctl -a | grep net.core.wmem_default
  • /sbin/sysctl -a | grep net.core.wmem_max
Compare it to:
  • fs.suid_dumpable = 1
  • fs.aio-max-nr = 1048576
  • fs.file-max = 6815744
  • kernel.shmall = 2097152
  • kernel.shmmax = 536870912
  • kernel.shmmni = 4096
  • kernel.sem = 250 32000 100 128
  • net.ipv4.ip_local_port_range = 9000 65500
  • net.core.rmem_default = 262144
  • net.core.rmem_max = 4194304
  • net.core.wmem_default = 262144
  • net.core.wmem_max = 1048586
In my image, I had to add the following lines to sysctl.conf:
  • vi /etc/sysctl.conf
  • # Added by a Monkey
  • fs.suid_dumpable = 1
  • fs.aio-max-nr = 1048576
  • fs.file-max = 6815744
  • kernel.shmmni = 4096
  • kernel.sem = 250 32000 100 128
  • net.ipv4.ip_local_port_range = 9000 65500
  • net.core.rmem_default = 262144
  • net.core.rmem_max = 4194304
  • net.core.wmem_default = 262144
  • net.core.wmem_max = 1048586
Well, obviously not the first bullet point ;)

Configure the rest for oracle in security limits:
  • vi /etc/security/limits.conf
Added:
  • oracle              soft    nproc   2047
  • oracle              hard    nproc   16384
  • oracle              soft    nofile  1024
  • oracle              hard    nofile  65536
  • oracle              soft    stack   10240
Apply the changes:
  • /sbin/sysctl -p
Done so far!

Downloading and Installing Oracle 11g XE
Create download directory:
  • su - oracle
  • mkdir /u01/download
  • cd /u01/download
I couldn't get wget to download XE, so I had to resort to a cheat:
  • Log into oracle.com on your HOST machine with a REAL browser
  • Find download on oracle.com
  • Accept licence agreement
  • Click link to download
  • Capture and copy request (network tab in Chrome will give you that):
    http://download.oracle.com/otn/linux/oracle11g/xe/oracle-xe-11.2.0-1.0.x86_64.rpm.zip?AuthParam=<somestring>
  • Paste it into ssh session:
  • wget http://download.oracle.com/otn/linux/oracle11g/xe/oracle-xe-11.2.0-1.0.x86_64.rpm.zip?AuthParam=<someString>  -O oracle-xe-11.2.0-1.0.x86_64.rpm.zip
  • And the file will be downloaded
I had some initial problems with this, and had to go back to the browser, retrace my steps for yet another download and get a new request from the browser. Worked the second time, don't know why. Maybe because I downloaded with wget while downloading at the same time from my aws instance..?

Installing is much easier:
  • su - root
  • unzip oracle-xe-11.2.0-1.0.x86_64.rpm.zip
  • cd /u01/download/Disk1
  • rpm -ivh oracle-xe-11.2.0-1.0.x86_64.rpm
  • /etc/init.d/oracle-xe configure
  • Accept the defaults (unless you know what you are doing!)
And you are done installing Oracle 11g XE! If you want to test APEX, just open port 8080 in the security group in the AWS EC2 Console, and use the public dns for the instance to reach it.

Configure Oracle User Environment
Just a touch if you are going to use oracle user in ssh later:
  • su - oracle
  • cat /u01/app/oracle/product/11.2.0/xe/bin/oracle_env.sh
  • <copy the lines below into the .bash_profile>
  • export ORACLE_HOME=/u01/app/oracle/product/11.2.0/xe
  • export ORACLE_SID=XE
  • export NLS_LANG=`$ORACLE_HOME/bin/nls_lang.sh`
  • export PATH=$ORACLE_HOME/bin:$PATH
  • cd
  • vi .bash_profile
  • <paste the lines above>
  • :wq
Tying it Up
At this point, you should restart to verify that everything works, create a new AMI based on the current state of the instance, enable ssh connect with oracle user, reconfigure swap space, etc, etc.

Remember not to use EBS storage for anything you really care for, data should be backed up in a safe manner (as you normally would do anyway).

29 comments:

  1. Fantastic guide to installing XE on AWS. It worked for me! One area that could do with a bit more explanation is the EC2 tools config. The pk and cert files have nothing to do with the key pair you set up for SSH, but are to be found in the AWS Console under My Account => Security credentials. You can download them from there to your local machine then copy them over to the server.

    ReplyDelete
  2. Another thought ... you probably do not need a 30GB volume to hold XE. The download zip is approx 300MB so obviously it unpacks into something bigger - say 1GB max. Then you have a max of 11GB user data on XE - though this will surely map into somewhat greater disk space. And you do have to keep some empty space. My guess is that if you are not planning to hold any great volume of data you will be fine with your standard 8GB volume. Even for a big database 20GB is probably plenty. And of course this matters as the AWS 'free tier' only allows you a max of 30GB! Having said this I'm willing to be corrected ...

    ReplyDelete
  3. And another ... yes, you can get APEX going at :8080/apex/ . But then you have to enter a workspace, username and password. If all else fails try using Internal, Admin and the SYSTEM password you specified during install.

    ReplyDelete
  4. @Unknown

    1. That is true, ec2 tools requires your aws credentials, your aws instance requires the key pair name you launched the instance with.

    2. But I did need 30Gb :-) This was just notes during my own install with my own requirements. Good point though.

    3. I would not recommend epg for anything approaching production related, use the apex listener in a Tomcat/Glassfish/... instead. Or at least a http server as a reverse proxy in front. If you open 8080, you should narrow which ip addresses allowed to connect to that port through the aws console.

    ReplyDelete
  5. Håvard, thank you. This walk-through was very helpful. Thanks for taking the time to post it.

    ReplyDelete
  6. Many Thanks for the detailed blog post. I was able to install and can execute sqlplus "/ as sysdba" from within the oracle account. I set up access to ports 1521 and 8080 in the corresponding security group in the AWS console and assigned an elastic IP address to the instance but somehow still cannot access to APEX externally (I tried using both the public DNA and the IP). I even executed

    chkconfig iptables off
    service iptables stop

    as suggested in http://jeffkemponoracle.com/2011/09/08/installing-oracle-11gxe-on-amazon-elastic-cloud/

    and

    SQL> EXEC DBMS_XDB.SETLISTENERLOCALACCESS(FALSE);

    as suggested in http://www.beyondoracle.com/2011/04/04/installing-oracle-xe-11g-r2-amazon-micro-instance-oel-5-4/

    but still have no success. Do you, perhaps, have any advice?

    ReplyDelete
  7. This comment has been removed by the author.

    ReplyDelete
  8. My mistake was replacing the loopback entry in /etc/hosts rather than appending the host name (which is what you suggest). The correct line should be:

    127.0.0.1 localhost localhost.localdomain [HOST NAME]

    Your advice was spot on.

    ReplyDelete
  9. Thanks for the detailed blog post.

    just want to add details space after installation Oracle XE 11.2 on default Amazon Linux Micro Instance 64 bit 8GB HDD. It looks that I have 4 Gb space after the installation:

    [oracle@ip-**-**-**-** ~]$ df -h
    Filesystem Size Used Avail Use% Mounted on
    /dev/xvda1 7.9G 3.9G 4.0G 50% /
    tmpfs 298M 156M 142M 53% /dev/shm

    ReplyDelete
  10. I followed this guide and it was very helpful. Thank you! The one additional step I had to discover was to make sure that the server's host name and IP were in /etc/hosts. Without that, /etc/init.d/oracle-xe configure would fail with this error message:

    cloneDBCreation.log:ORA-27101: shared memory realm does not exist
    cloneDBCreation.log:ORA-00119: invalid specification for system parameter LOCAL_LISTENER
    cloneDBCreation.log:ORA-00130: invalid listener address '(ADDRESS=(PROTOCOL=TCP)(HOST=${yourHostName})(PORT=1521))'

    After adding my host name to /etc/hosts the database configuration script ran successfully. I hope this note helps others.

    ReplyDelete
  11. I love the way you write and share your niche! Very interesting and different! Keep it coming!
    Amazon

    ReplyDelete
  12. Here i had read the content you had posted. It is much interesting so please keep update like this.


    Best Dental Clinic in Alwarpet
    Cosmetic Dentistry
    Root Canal Treatment In Chennai
    Teeth Whitening treatment

    ReplyDelete
  13. This idea is mind blowing. I think everyone should know such information like you have described on this post. Thank you for sharing this explanation.Your final conclusion was good. We are sowing seeds and need to be patiently wait till it blossoms.
    Software Testing Training in Chennai
    SEO Training in Chennai
    Informatica Training in Chennai
    Digital Marketing Training in Chennai

    ReplyDelete
  14. Thanks for the informative article. This is one of the best resources I have found in quite some time. Nicely written and great info. I really cannot thank you enough for sharing.

    Wellnesscentres in Chennai
    Weightloss in Chennai
    Weightgain in Chennai

    ReplyDelete
  15. I just stumbled upon your blog and wanted to say that I have really enjoyed reading your blog posts. Any way I’ll be subscribing to your feed and I hope you post again soon.

    Manufacturing ERP
    Human Resources Management Software
    CCTV Camera Dealers in Chennai

    ReplyDelete
  16. I simply want to say I’m very new to blogs and actually loved you’re blog site. Almost certainly I’m going to bookmark your blog post . You absolutely come with great well written articles. Thanks a lot for sharing your blog.
    Economics Journal
    Journal Of Ecology
    Thesis by publication
    National Journal
    Journal Of Information Technology

    ReplyDelete
  17. All the latest updates from the PythonAutomationminds team. Python Automationminds lets you program in Python,

    in your browser. No need to install any software, just start coding straight away. There's a fully-functional web-based

    console and a programmer's text-editor
    Phyton training in Chennai


    ReplyDelete
  18. Learn about other Apache projects that are part of the Hadoop ecosystem, including Pig, Hive, HBase, ZooKeeper, Oozie, Sqoop, Flume, among others. Big Data University provides separate courses on these other projects, but we recommend you start here.

    Bigdata training in Chennai OMR

    ReplyDelete
  19. I have read this post. collection of post is a nice one AWS Online Training

    ReplyDelete
  20. Thanks for the informative article. This is one of the best resources I have found in quite some time. Nicely written and great info. I really cannot thank you enough for sharing.

    Best Astrologer In India

    Astrologer In India

    Top Astrologer In India

    Best Numerologist In India

    ReplyDelete
  21. Very informative! Thanks for providing your information your valuable information. Keep updating with us AWS Online Course

    ReplyDelete
  22. Needed to compose you a very little word to thank you yet again regarding the nice suggestions you’ve contributed here.
    erp in chennai

    Education erp software in chennai

    ReplyDelete
  23. Its really an Excellent post. I just stumbled upon your blog and wanted to say that I have really enjoyed reading your blog. Thanks for sharing....

    Solar Rooftop
    Solar Water Heater
    Solar Panel
    Solar Module
    Energy Efficient
    BLDC Fan
    Solar Power
    Power Plant
    Solar Training
    Solar Pump

    ReplyDelete
  24. Your site is amazing and your blogs are informative and knlowledgeble to my websites.This is one of the best tips in my life.I have in quite some time.Nicely written and great info.Thanks to share the more informations.

    Seo Experts
    Seo Company
    Web Designing Company
    Digital Marketing
    Web Development Company
    App Development

    ReplyDelete

  25. Awesome article. It is so detailed and well formatted that i enjoyed reading it as well as get some new information too. provides you with a state of the art software which combines modern GPU technology (Graphic Processing Units) with the best practices in today’s Big Data platforms, providing up to 100x faster insights from data.
    Bigdata Training in Chennai OMR

    ReplyDelete
  26. Robotic Process Automation (RPA) is one of the most exciting developments in Business Process Management (BPM) in recent history. Some industry experts believe it may be even more transformational than cloud computing transformational than cloud Automationminds team. (RPA)Automationminds lets you program in (RPA),
    Robotic Process Automation course

    ReplyDelete