Sunday, March 18, 2012

Installfest: Oracle 11g XE on Amazon Linux Micro Instance

This is just a quick (as in "I really, really mean it quick") note on installing Oracle 11g XE on Amazon EC2 on a vanilla Amazon Linux AMI as a micro instance. Cloudy...

Why Amazon Linux? It's easy and it's pre configured with most of what you need. Both cloud wise and for other things. It also gets good reviews as a production OS.

No, Oracle 11g XE on Amazon Linux this is not supported by Oracle. But then again, you will receive no official support on Oracle 11g XE by Oracle on any OS...

I have not validated if every step described below is necessary, by I reasoned that the standard install steps will stand, even if they upgrade the default Amazon Linux AMI.

If you have no experience with Linux or Amazon EC2, I think you will be better off with a more verbose installation guide than this is!

Create the Instance
If I am unclear (or just a tad abbreviated), just follow this documentation at any time.
  • Launch new instance, and choose "Basic Amazon Linux AMI YYYY.MM"
  • Choose instance type micro, set or create key pairs and set or create security group (everything else, just keep default)
Your are done creating the instance.

Connect to the Instance
I use PuTTY, for alternatives or a more elaborate description of how to connect with PuTTY, see this documentation. I assume you have PuTTY suite (with PuTTYGen) installed.

You also have to check that port 22 is open for the security group associated with the image.
  • Start PuTTYGen
  • Load the .pem file created from then key pair in use with the instance (switch to "all files" to see it)
  • Click Save Private Key, and Yes
  • Exit PuTTYGen
  • Start PuTTY
  • Copy Public DNS from EC2 Console
  • Paste Public DNS into Host Name in PuTTY Configuration
  • On left side, go to Connection -> SSH -> Auth and browse to the location of the previously generated ppk-file
  • Back to Session, give it a name  in "Saved Sessions" and Save
  • Click Open and Yes in the dialog
  • Connect as ec2-user
You are done connecting to your new instance.

Configure EC2 Tools
This you can do on your host machine, or the newly created instance. I will describe how for the newly created instance (less work for me, but also less secure for you!).
  • Log in as ec2-user
Configure root user:
  • sudo su - root
  • passwd <new password>
  • exit
  • su - root
  • <new password>
  • id
EC2 Tools are installed by default, but you have to let it know which credentials to use. Copy the pk and cert files into /root directory, and modify .bash_profile:
  • su - root
  • vi ./.bash_profile
  • export EC2_PRIVATE_KEY=/root/pk-<your pk>.pem
  • export EC2_CERT=/root/cert-<your pem>.pem
  • export EC2_URL=https://ec2.<region>.amazonaws.com
I use eu-west-1 as region. You should now be able to issue
  • ec2-describe-availability-zones
And receive a list of zones.

Configure Instance Storage
First, create and attach a new volume (30Gb) to the instance:
  • su - root 
  • ec2-describe-instances 
  • Take a not of the instance id and availability zone 
  • ec2-create-volume --size 30 --availability-zone <zoneSameAsTheInstance>
  • Take a note of the volume created
  • ec2-attach-volume <yourVolumeId> -i <yourInstanceId> -d <yourDeviceName>
  • Like this: ec2-attach-volume vol-xxxxxx -i i-xxxxxx -d /dev/sdf
  • ec2-describe-volumes
Format volume and make it stick on reboot:
  • mkfs -t ext3 /dev/sdf
  • tune2fs -m0 /dev/sdf
  • mkdir /u01
  • mount /dev/sdf /u01
  • vi /etc/fstab
  • Add the following line:
  • /dev/sdf    /u01        auto    defaults,noatime 0 0
Increase swap file (using the root device):
  • dd if=/dev/zero of=/home/swapfile bs=1024 count=2097152
  • mkswap /home/swapfile
  • vi /etc/fstab
  • Add the following line:
  • /home/swapfile swap swap defaults 0 0
This is a bit of a cheat, because the default swap disk will still attach, you cat alter that later.

Set host name for the server:
  • vi /etc/sysconfig/network
  • HOSTNAME=<yourHostName>
  • :wq
  • vi /etc/hosts
  • Add  <yourHostName> to the line containing 127.0.0.1
  • :wq
Restart to check if everything is OK:
  • shutdown -r now
At this stage I created a new AMI based on the current configuration (with the new 30Gb volume still attached). You never know :-)

Configure Instance for Oracle 11g XE
Since we can't use yum oracle-validated on the Amazon Linux, we will have to do this the hard way.

Create oracle user++:
  • su - root
  • groupadd oinstall
  • groupadd dba
  • useradd -g oinstall -G dba,oinstall oracle
  • passwd oracle
  • chown oracle:oinstall /u01
Get the latest packages:
  • yum install binutils-2.*
  • yum install compat-libstdc++-33*
  • yum install elfutils-libelf-0.*
  • yum install elfutils-libelf-devel-*
  • yum install gcc-4.*
  • yum install gcc-c++-4.*
  • yum install glibc-2.*
  • yum install glibc-common-2.*
  • yum install glibc-devel-2.*
  • yum install glibc-headers-2.*
  • yum install ksh-2*
  • yum install libaio-0.*
  • yum install libaio-devel-0.*
  • yum install libgcc-4.*
  • yum install libstdc++-4.*
  • yum install libstdc++-devel-4.*
  • yum install make-3.*
  • yum install sysstat-7.*
  • yum install unixODBC-2.*
  • yum install unixODBC-devel-2.*
Check the parameter settings:
  • /sbin/sysctl -a | grep fs.suid_dumpable
  • /sbin/sysctl -a | grep fs.aio-max-nr
  • /sbin/sysctl -a | grep fs.file-max
  • /sbin/sysctl -a | grep kernel.shmall
  • /sbin/sysctl -a | grep kernel.shmmax
  • /sbin/sysctl -a | grep kernel.shmmni
  • /sbin/sysctl -a | grep kernel.sem
  • /sbin/sysctl -a | grep net.ipv4.ip_local_port_range
  • /sbin/sysctl -a | grep net.core.rmem_default
  • /sbin/sysctl -a | grep net.core.rmem_max
  • /sbin/sysctl -a | grep net.core.wmem_default
  • /sbin/sysctl -a | grep net.core.wmem_max
Compare it to:
  • fs.suid_dumpable = 1
  • fs.aio-max-nr = 1048576
  • fs.file-max = 6815744
  • kernel.shmall = 2097152
  • kernel.shmmax = 536870912
  • kernel.shmmni = 4096
  • kernel.sem = 250 32000 100 128
  • net.ipv4.ip_local_port_range = 9000 65500
  • net.core.rmem_default = 262144
  • net.core.rmem_max = 4194304
  • net.core.wmem_default = 262144
  • net.core.wmem_max = 1048586
In my image, I had to add the following lines to sysctl.conf:
  • vi /etc/sysctl.conf
  • # Added by a Monkey
  • fs.suid_dumpable = 1
  • fs.aio-max-nr = 1048576
  • fs.file-max = 6815744
  • kernel.shmmni = 4096
  • kernel.sem = 250 32000 100 128
  • net.ipv4.ip_local_port_range = 9000 65500
  • net.core.rmem_default = 262144
  • net.core.rmem_max = 4194304
  • net.core.wmem_default = 262144
  • net.core.wmem_max = 1048586
Well, obviously not the first bullet point ;)

Configure the rest for oracle in security limits:
  • vi /etc/security/limits.conf
Added:
  • oracle              soft    nproc   2047
  • oracle              hard    nproc   16384
  • oracle              soft    nofile  1024
  • oracle              hard    nofile  65536
  • oracle              soft    stack   10240
Apply the changes:
  • /sbin/sysctl -p
Done so far!

Downloading and Installing Oracle 11g XE
Create download directory:
  • su - oracle
  • mkdir /u01/download
  • cd /u01/download
I couldn't get wget to download XE, so I had to resort to a cheat:
  • Log into oracle.com on your HOST machine with a REAL browser
  • Find download on oracle.com
  • Accept licence agreement
  • Click link to download
  • Capture and copy request (network tab in Chrome will give you that):
    http://download.oracle.com/otn/linux/oracle11g/xe/oracle-xe-11.2.0-1.0.x86_64.rpm.zip?AuthParam=<somestring>
  • Paste it into ssh session:
  • wget http://download.oracle.com/otn/linux/oracle11g/xe/oracle-xe-11.2.0-1.0.x86_64.rpm.zip?AuthParam=<someString>  -O oracle-xe-11.2.0-1.0.x86_64.rpm.zip
  • And the file will be downloaded
I had some initial problems with this, and had to go back to the browser, retrace my steps for yet another download and get a new request from the browser. Worked the second time, don't know why. Maybe because I downloaded with wget while downloading at the same time from my aws instance..?

Installing is much easier:
  • su - root
  • unzip oracle-xe-11.2.0-1.0.x86_64.rpm.zip
  • cd /u01/download/Disk1
  • rpm -ivh oracle-xe-11.2.0-1.0.x86_64.rpm
  • /etc/init.d/oracle-xe configure
  • Accept the defaults (unless you know what you are doing!)
And you are done installing Oracle 11g XE! If you want to test APEX, just open port 8080 in the security group in the AWS EC2 Console, and use the public dns for the instance to reach it.

Configure Oracle User Environment
Just a touch if you are going to use oracle user in ssh later:
  • su - oracle
  • cat /u01/app/oracle/product/11.2.0/xe/bin/oracle_env.sh
  • <copy the lines below into the .bash_profile>
  • export ORACLE_HOME=/u01/app/oracle/product/11.2.0/xe
  • export ORACLE_SID=XE
  • export NLS_LANG=`$ORACLE_HOME/bin/nls_lang.sh`
  • export PATH=$ORACLE_HOME/bin:$PATH
  • cd
  • vi .bash_profile
  • <paste the lines above>
  • :wq
Tying it Up
At this point, you should restart to verify that everything works, create a new AMI based on the current state of the instance, enable ssh connect with oracle user, reconfigure swap space, etc, etc.

Remember not to use EBS storage for anything you really care for, data should be backed up in a safe manner (as you normally would do anyway).

28 comments:

  1. Fantastic guide to installing XE on AWS. It worked for me! One area that could do with a bit more explanation is the EC2 tools config. The pk and cert files have nothing to do with the key pair you set up for SSH, but are to be found in the AWS Console under My Account => Security credentials. You can download them from there to your local machine then copy them over to the server.

    ReplyDelete
  2. Another thought ... you probably do not need a 30GB volume to hold XE. The download zip is approx 300MB so obviously it unpacks into something bigger - say 1GB max. Then you have a max of 11GB user data on XE - though this will surely map into somewhat greater disk space. And you do have to keep some empty space. My guess is that if you are not planning to hold any great volume of data you will be fine with your standard 8GB volume. Even for a big database 20GB is probably plenty. And of course this matters as the AWS 'free tier' only allows you a max of 30GB! Having said this I'm willing to be corrected ...

    ReplyDelete
  3. And another ... yes, you can get APEX going at :8080/apex/ . But then you have to enter a workspace, username and password. If all else fails try using Internal, Admin and the SYSTEM password you specified during install.

    ReplyDelete
  4. @Unknown

    1. That is true, ec2 tools requires your aws credentials, your aws instance requires the key pair name you launched the instance with.

    2. But I did need 30Gb :-) This was just notes during my own install with my own requirements. Good point though.

    3. I would not recommend epg for anything approaching production related, use the apex listener in a Tomcat/Glassfish/... instead. Or at least a http server as a reverse proxy in front. If you open 8080, you should narrow which ip addresses allowed to connect to that port through the aws console.

    ReplyDelete
  5. Håvard, thank you. This walk-through was very helpful. Thanks for taking the time to post it.

    ReplyDelete
  6. Many Thanks for the detailed blog post. I was able to install and can execute sqlplus "/ as sysdba" from within the oracle account. I set up access to ports 1521 and 8080 in the corresponding security group in the AWS console and assigned an elastic IP address to the instance but somehow still cannot access to APEX externally (I tried using both the public DNA and the IP). I even executed

    chkconfig iptables off
    service iptables stop

    as suggested in http://jeffkemponoracle.com/2011/09/08/installing-oracle-11gxe-on-amazon-elastic-cloud/

    and

    SQL> EXEC DBMS_XDB.SETLISTENERLOCALACCESS(FALSE);

    as suggested in http://www.beyondoracle.com/2011/04/04/installing-oracle-xe-11g-r2-amazon-micro-instance-oel-5-4/

    but still have no success. Do you, perhaps, have any advice?

    ReplyDelete
  7. This comment has been removed by the author.

    ReplyDelete
  8. My mistake was replacing the loopback entry in /etc/hosts rather than appending the host name (which is what you suggest). The correct line should be:

    127.0.0.1 localhost localhost.localdomain [HOST NAME]

    Your advice was spot on.

    ReplyDelete
  9. Thanks for the detailed blog post.

    just want to add details space after installation Oracle XE 11.2 on default Amazon Linux Micro Instance 64 bit 8GB HDD. It looks that I have 4 Gb space after the installation:

    [oracle@ip-**-**-**-** ~]$ df -h
    Filesystem Size Used Avail Use% Mounted on
    /dev/xvda1 7.9G 3.9G 4.0G 50% /
    tmpfs 298M 156M 142M 53% /dev/shm

    ReplyDelete
  10. Thank you for sharing such a nice and interesting blog with us. I have seen that all will say the same thing repeatedly. But in your blog, I had a chance to get some useful and unique information. I would like to suggest your blog in my dude circle.
    Jobs in Chennai
    Jobs in Bangalore
    Jobs in Delhi
    Jobs in Hyderabad
    Jobs in Kolkata
    Jobs in Mumbai
    Jobs in Noida
    Jobs in Pune

    ReplyDelete
  11. Thank you for taking the time to provide us with your valuable information. We strive to provide our candidates with excellent care and we take your comments to heart.As always, we appreciate your confidence and trust in us
    Digital Marketing Company in India

    ReplyDelete
  12. I am not sure the place you are getting your information, however good topic.I needs to spend some time studying more or understanding more.Thank you for wonderful information I was in search of this info for my mission.

    HR Consultancy in Chennai
    Recruitment Consultancy in Chennai
    Manpower Consultancy in Chennai

    ReplyDelete
  13. I just see the post i am so happy the post of information's.So I have really enjoyed and reading your blogs for these posts.Any way I’ll be subscribing to your feed and I hope you post again soon.

    Digital Marketing Company in Chennai

    ReplyDelete
  14. I followed this guide and it was very helpful. Thank you! The one additional step I had to discover was to make sure that the server's host name and IP were in /etc/hosts. Without that, /etc/init.d/oracle-xe configure would fail with this error message:

    cloneDBCreation.log:ORA-27101: shared memory realm does not exist
    cloneDBCreation.log:ORA-00119: invalid specification for system parameter LOCAL_LISTENER
    cloneDBCreation.log:ORA-00130: invalid listener address '(ADDRESS=(PROTOCOL=TCP)(HOST=${yourHostName})(PORT=1521))'

    After adding my host name to /etc/hosts the database configuration script ran successfully. I hope this note helps others.

    ReplyDelete

  15. Wonderful blog.. Thanks for sharing informative Post. Its very useful to me.

    Installment loans
    Payday loans
    Title loans

    ReplyDelete
  16. Really Good blog post.provided a helpful information.I hope that you will post more updates like this.
    Digital marketing company in Chennai

    ReplyDelete
  17. It’s the best time to make some plans for the future and it is time to be happy. I’ve read this post and if I could I want to suggest you few interesting things or suggestions.You can write next articles referring to this article. I desire to read even more things about it..
    AWS Training in Chennai
    Hadoop Training in Chennai
    Software Testing Training in Chennai

    ReplyDelete
  18. I love the way you write and share your niche! Very interesting and different! Keep it coming!
    Amazon

    ReplyDelete
  19. This information is impressive; I am inspired with your post writing style & how continuously you describe this topic.
    Web Design Company in Chennai

    ReplyDelete
  20. i really like this blog.And i got more information's from this blog.thanks for sharing!!!!Digital Marketing Company in Chennai

    ReplyDelete
  21. Installing Oracle 11g XE on amazon linux micro instance is very well explained with the help of commands which are easier to remember as well implement more easily.


    Android app development company in chennai

    ReplyDelete
  22. Thanks for spending your valuable time in delivering the most valuable content here.I loved the way you write and suggest my friends too for getting aware of your blogs.
    Pawn Shops in Montgomery
    Pawn Shops in Birmingham
    Pawn Shops in Mobile

    ReplyDelete
  23. Thank you for your sharing information..waiting for more information.. very happy to visit your post..
    SAP HR Training in Chennai
    SAP ABAP Training in Chennai
    SAP FICO Training in Chennai

    ReplyDelete

  24. Its a wonderful post and very helpful, thanks for all this information. You are including better information regarding this topic in an effective way.Thank you so much

    Installment Loans Near Me
    Title loans Near Me
    Cash Advances Near Me

    ReplyDelete
  25. I’ve been browsing on-line greater than three hours today, but I never discovered any attention-grabbing article like yours. It is beautiful worth sufficient for me. Personally, if all webmasters and bloggers made good content material as you did, the net will be a lot more helpful than ever before.
    Vitiligo Treatment
    Dry Skin Treatment

    ReplyDelete
  26. this is very nice blog this studying course information very useful to everyone who have learning this information.

    Informatica Training in Chennai

    ReplyDelete
  27. A nice article here, i think that people who have grown up with the idea of using computers are showing more responsibility towards writing posts that are thoughtful, do not have grammar mistakes and pertinent to the post..
    Home Interior Designers in Chennai
    Home Interiors in Chennai

    ReplyDelete