Friday, July 1, 2016

AWS EC2 Start/Stop Scripts using Tagging

So my first version of this just used a list of ec2 ids and stopped and started.  I need it to make it more "cloud like".  So what is more cloud like? Tagging!  I now have it building a list based on an explicit tag, then comparing it to another list that is dynamic based on a tag called shutdown with the value of Exclude.  Anything with the exclude value, I don't want it shut down.  As always, anyone has ways of making this better, post below.


instance_list=$(aws ec2 describe-instances --filters Name=tag-value,Values="Enter-tag-value-here-no-quotes"| cut -f 8 | grep ^i-)

exclude_list=$(aws ec2 describe-instances --filters Name=tag-value,Values="Enter-tag-value-here-no-quotes"| cut -f 8 | grep ^i-)

while IFS= read -r instance_id
        echo "Shutting down: $instance_id";
        if [[ ${exclude_list[*]} =~ $instance_id ]]; then
   echo "The instance must stay powered on";
            status=$(aws ec2 describe-instances --instance-ids $instance_id | cut -f 3 | grep 'stopped\|stopping\|running\|pending')

            if [ $status != "stopped" ];
                aws ec2 stop-instances --instance-ids $instance_id
                sleep 10
                status=$(aws ec2 describe-instances --instance-ids $instance_id | cut -f 3 | grep 'stopped\|stopping\|running\|pending')
                echo "The $instance_id is $status";
done <<< "$instance_list"


instance_list=$(aws ec2 describe-instances --filters Name=tag-value,Values="Enter-tag-value-here-no-quotes" | cut -f 8 | grep ^i-)

while IFS= read -r instance_id
        status=$(aws ec2 describe-instances --instance-ids $instance_id | cut -f 3 | grep 'stopped\|stopping\|running\|pending')
        echo "The instance $instance_id is $status"

        while [ "$status" != 'running' ];
                aws ec2 start-instances --instance-ids $instance_id
                sleep 10
                status=$(aws ec2 describe-instances --instance-ids $instance_id | cut -f 3| grep 'stopped\|stopping\|running\|pending')
                echo "The instance $instance_id is $status"

                if [ "$status" == 'running' ]; then
                        echo "The $instance_id is $status"
done <<< "$instance_list"

Thursday, June 30, 2016

AWS EC2 Startup/Shutdown Scripts

Couple assumptions:

1.  You've already installed the aws cli
2.  You've already run aws configure
3.  You are running this from an instance that never gets shutdown




while IFS= read -r instance_id
        status=$(aws ec2 describe-instances --instance-ids $instance_id | cut -f 3 | grep stopped)
        echo "The instance $instance_id is $status"

        while [ "$status" != 'stopped' ];
                aws ec2 stop-instances --instance-ids $instance_id
                sleep 10
                status=$(aws ec2 describe-instances --instance-ids $instance_id | cut -f 3| grep stopped)
                echo "The instance $instance_id is $status"

                if [ "$status" == 'stopped' ]; then
                        echo "The $instance_id is $status"
done <<< "$instance_list"




while IFS= read -r instance_id
        status=$(aws ec2 describe-instances --instance-ids $instance_id | cut -f 3 | grep running)
        echo "The instance $instance_id is $status"

        while [ "$status" != 'running' ];
                aws ec2 start-instances --instance-ids $instance_id
                sleep 10
                status=$(aws ec2 describe-instances --instance-ids $instance_id | cut -f 3| grep running)
                echo "The instance $instance_id is $status"

                if [ "$status" == 'running' ]; then
                        echo "The $instance_id is $status"
done <<< "$instance_list"

Right now this is doing it based on the list you write.  I did out that if you could grab all instances like this:

instance_list=$(aws ec2 describe-instances | cut -f 8 | grep ^i-)

The problem is that your shutting down the box with the scripts.  Plus I do have a couple of servers I don't want to go down.  Good enough for now but I need to add a way to exclude instances and also save EIPs so I can reattach the same EIP when I bring it back up.  If you know how to do it, please feel free to improve this in the comments!

Monday, April 18, 2016

How to move the storage location of Docker in CentOS/Red Hat 7

  1. systemctl stop docker
  2. verify its off ps -ef | grep docker
  3. edit the config file
    - vim /etc/sysconfig/docker
  4. modify the following line to:
    -OPTIONS='--selinux-enabled -g /path/to/your/folder'
  5. Save and close
  6. systemctl start docker
You should now see folders in the new folder location.

Tested on: CentOS 7.2

Source:  Link

Wednesday, December 2, 2015

Setting up an C-ICAP Server using the "The c-icap project"

Couple things first:

Download info:
  • Main Software -
  • Modules -
I built this from multiple articles that are linked some where on here.  Not one had all the answers.  Here is my brain dump from it.

Download c-icap and c-icap modules

tar xvf c_icap-x.x.x.tar.gz
cd c_icap-x.x.x
./configure --prefix=/usr/local/c-icap
make install 
vim /usr/local/c-icap/etc/c-icap.conf 

Under ServerAdmin: Enter your email

Under ServerName: Enter your hostname of the server

The ModulesDir is wrong and needs to be changed to:
  • Wrong: ModulesDir /usr/local/c-icap/lib/c_icap
  • Correct: ModulesDir /usr/local/lib/c_icap

The ServicesDir is wrong and needs to be changed to: 
  • Wrong: ServicesDir /usr/local/c-icap/lib/c_icap
  • Correct: ServicesDir /usr/local/lib/c_icap
 The TemplateDir  is wrong also and needs to be changed to:
  • Wrong: TemplateDir /usr/local/c-icap/share/c_icap/templates/ 
  • Correct: TemplateDir /usr/local/share/c_icap/templates/
 Next Step, test your server:

/usr/local/c-icap/bin/c-icap -N -D -d 10

Instructions: Link

If everything is correct, it should start up.  Kill it and we will continue. Next we need to configure the module to scan using clamav.  This is assuming you already have clamav installed and working.

If you want to create a c-icap service do the following:
vi /etc/rc.d/init.d/c-icap

# bin/bash
# c-icap: Start/Stop c-icap
# chkconfig: - 70 30
# description: c-icap is an implementation of an ICAP server.
# processname: c-icap
# pidfile: /var/run/c-icap/

. /etc/rc.d/init.d/functions
. /etc/sysconfig/network


start() {
   echo -n $"Starting c-icap: "
   daemon /usr/local/c-icap/bin/c-icap -f $CONFIG_FILE
   [ $RETVAL -eq 0 ] && touch /var/lock/subsys/c-icap
   return $RETVAL
stop() {
   echo -n $"Stopping c-icap: "
   killproc c-icap
   rm -f /var/run/c-icap/c-icap.ctl
   [ $RETVAL -eq 0 ] && rm -f $PID_DIR/ /var/lock/subsys/c-icap
   return $RETVAL
case "$1" in
      status c-icap
      echo $"Usage: $0 {start|stop|status|restart}"
   exit 1
exit $?

This was taken from one of the links below but I had to modify it to make it work with changes in a recent update to the project.

Test that it starts:
 /etc/rc.d/init.d/c-icap start 

Create the final pieces to the service:
chkconfig --add c-icap
chkconfig c-icap on 

We need to extract the c-icap modules

tar xvf c_icap_modules-x.x.x.tar.gz
cd c_icap-x.x.x
./configure --with-c-icap=/usr/local/c-icap --prefix=/usr/local/c-icap
make install 
vim /usr/local/c-icap/etc/c-icap.conf 

Now we need to add the virus scan module.  You already have the c-icap config open so at the end of the file add:
  • Include /usr/local/etc/virus_scan.conf
Instructions:  Link

The virus_scan.conf needs to me modified.

vim /usr/local/etc/virus_scan.conf

Add to the bottom:

Include /usr/local/etc/clamd_mod.conf

I also changed the one line to:
virus_scan.DefaultEngine clamav

The clamd needs to modified for the correct Socket Location:
vim /usr/local/etc/clamd_mod.conf

Change the Socket Location to:
clamd_mod.ClamdSocket /var/run/clamav/clamd.sock

service c-icap restart

You can test it by running the following commands:

No Virus Test File: 
/usr/local/c-icap/bin/c-icap-client -f /bin/ls  \
            -s "srv_clamav?allow204=on&force=on&sizelimit=off&mode=simple"

Virus Test File:
/usr/local/c-icap/bin/c-icap-client -f /usr/local/share/clamav-0.x.x/test/clam.exe \
   -s "srv_clamav?allow204=on&force=on&sizelimit=off&mode=simple"

Log Files are located:
  • ServerLog /usr/local/c-icap/var/log/server.log 
  • AccessLog /usr/local/c-icap/var/log/access.log
A lot of the errors will show up in the server.log file.


ERROR: Unable to find specified template: /usr/local/p/share/c_icap/templates//virus_scan/en/VIRUS_FOUND

Means the path is wrong.

 clamd_connect: Can not connect to clamd server on /var/run/clamav/clamd.ctl!

This means that the Socket Location in clamav is different then what you have listed.

Check these two files for the location: /usr/local/etc/clamd.conf /etc/clamd.conf

Source:  Link Link
Virus Test: Link ******REMEMBER THESE ARE ACTUAL VIRUSES!!!!******

Tuesday, November 3, 2015

Joining CentOS/RHEL (6.x) to Active Directory (Windows Server Domain) [Updated]


Install the following packages

yum install pam_krb5 pam_ldap nss-pam-ldapd samba ntp

*If you already have any of these installed, it'll skip them.

First make a backup of the config.
cp /etc/krb5.conf /etc/krb5.conf.bak

 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

 default_realm = DOMAIN.COM
 dns_lookup_realm = false
 dns_lookup_kdc = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true

  admin_server = DC.DOMAIN.COM
  kdc = x.x.x.x

[domain_realm] = DOMAIN.COM = DOMAIN.COM

Again backup the config
cp /etc/samba/smb.conf /etc/samba/smb.conf.bak

You can also use system-config-authentication if you have a gui.

Note:  The workgroup is the left most part of the domain

workgroup = DOMAIN
password server = x.x.x.x
realm = DOMAIN.COM
security = ADS
winbind enum users = Yes
winbind enum groups = Yes
winbind use default domain = No
winbind separator = +
idmap config * : backend = autorid
idmap config * : range = 1000000-19999999
idmap config * : rngesize = 1000000
template homedir= /home/%D/%U
template shell = /bin/bash

If you don't want the users to have to type DOMAIN+username then remove the winbind separator line and change the winbind user default domain = yes.  Restart smb and winbind.

Again backup the config
cp /etc/nsswitch.conf /etc/nsswitch.conf.bak

passwd:     files winbind
group:      files winbind
shadow:     files winbind

Again backup the config
cp /etc/ntp.conf /etc/ntp.conf.bak

Note:  The is the ip of your server

Pick one of these ways:
1. authconfig-tui (select Winbind, click ok. Select Use Shadow Passwords, Use Winbind Authentication, Local authorization is sufficient)

2. Create the files manually.

Again backup the config
cp /etc/pam.d/system-auth /etc/pam.d/system-auth.bak

auth        required
auth        sufficient try_first_pass
auth        requisite uid >= 500 quiet
auth        sufficient use_first_pass
auth        sufficient use_first_pass
auth        required

account     required
account     required broken_shadow
account     sufficient
account     sufficient uid < 500 quiet
account     [default=bad success=ok user_unknown=ignore]
account     [default=bad success=ok user_unknown=ignore]
account     required

password    requisite difok=4 dcredit=-2 ucredit=-2 ocredit=-2 lcredit=-2 minlen=12 retry=3
password    sufficient sha512 shadow try_first_pass use_authtok
password    sufficient use_authtok
password    sufficient use_authtok
password    required

session     optional revoke
session     required
session     optional skel=/etc/skel umask=0644
session     [success=1 default=ignore] service in crond quiet use_uid
session     required
session     optional

auth       required
auth       include      password-auth
auth    sufficient
account    required
account    include      password-auth
password   include      password-auth
# close should be the first session rule
session    required close
session    required
# open should only be followed by sessions to be executed in the user context
session    required open env_params
session    optional force revoke
session    include      password-auth

Edit your /etc/hosts file
</etc/hosts> dc

Restart the following services:

service smb restart
service winbind restart
service sshd restart
service ntpd restart
chkconfig winbind on

*If any of those fail, you have something configured wrong.
Then run:  [root@node001user1]# kinit domainadmin@CENTOS.COM (THIS MUST BE IN ALL CAPS!)

Confirm the ticket was obtained: [root@node001 user1]# klist

Get the time from the server: [root@node001 user1]# net time

Sync the time from the server: [root@node001 user1]# ntpdate -u

Then run the following command to join it to the domain.

[root@node001 user1]# net ads join -U domainadmin (replace with your domain admin username)

Run some more tests:

wbinfo -t
wbinfo -u
wbinfo -g

getent passwd
getent groups

If any of those fail, something isn't configured correctly.

If you want the domain admins and admins to have privileged access, you need to add this to the bottom of your sudoers file.  You may have to over write the read only file to save it.

</etc/sudoers> ***NEED TO VERIFY IF THIS STILL WORKS****
[root@node001 user1]# cat /etc/sudoers

%BUILTIN\administrators ALL=(ALL) ALL
%"domain admins" ALL=(ALL) ALL

Some great additional trouble shooting commands can be found here -> Link

Also if you need to find your base dn to locate the group your user accounts are stored, I explain how to do that here -> Link

Another Great reference is here:  Link

If you have a Red Hat Subscription, they provide some good additional information:  Link

Integrating Red Hat Enterprise Linux 6 with Active Directory (Last updated 2014): Link


Tested: RHEL 6.4+Windows 2k8 R2, RHEL 6.4+Windows 2k12 R2

kinit: Cannot find KDC for requested realm while getting initial credentials

The error message doesn't really give you a clear "this is your problem" error.

Possible Problems
  1.  krb5.conf is wrong (see here for more information on that Link)
  2. kinit command is wrong. The case matters.
*if the domain name is not in ALL CAPS, you will get this error message.

How to Build a Yum Repository [Updated]

yum install yum-utils createrepo vsftpd

Create the directory: /var/ftp/pub/yum/*OSName*/*RELEASE*/*PATCH*/base/*ARCH*



mkdir -p /var/ftp/pub/yum/centos/6/6/base/x86_64

Option 1:  From Install CD

Required:  Installation Media

cp -ar /*CDMOUNTPOINT*/Packages/* /var/ftp/pub/yum/centos/6/6/base/x86_64
cp /*CDMOUNTPOINT*/repodata/*comps*.xml /opt/yum/rhel6.3/repodata/comps.xml

Option 2:  From an Internet Repo

Required:  Internet Connection

If you do it this way, you are grabbing the latest patches from a repository on the internet.

reposync --gpgcheck -l -p /etc/yum.repos.d/redhat-rhui.repo --download_path=/var/ftp/pub/yum/RHEL/7/1/base/x86_64 --downloadcomps --download-metadata

Source:  Link Link Link 

Server Config

service vsftpd start

*If running iptables, you will have to allow it through the firewall

cd /var/ftp/pub/yum/centos/6/6/base/x86_64
createrepo -v .

vim /etc/yum.reposb.d/custom.repo
name=Centos $releasever Base Updates 6.6
baseurl=ftp://*ip-of-the-computer*/pub/centos/6/6/base/x86_64 enable=1 gpgcheck=0 (this can be changed to 1 once the certs are installed on the system)

yum clean all
yum repolist

Your repo should now be listed

Couple of Notes:
SELinux and the firewall was turned off while doing this

Note:  Some systems may require you to run yum-arch /var/ftp/pub/yum/centos

Tested In: CentOS 4.3, 6.6, RHEL 6.4 (just change version numbers above to match your version of the OS)


repomd.xml: [Errno 14] PYCURL ERROR 9 - "Server denied you to change to the given directory" Trying other mirror."

After building your own yum repo you get the error: "repomd.xml: [Errno 14] PYCURL ERROR 9 - "Server denied you to change to the given directory" Trying other mirror."

You setup a server to be a repo server.  You've installed ftp and everything seems to be correct.  You've also checked the permissions on all the files and they are correct at 755.  Always test the ftp address in a browser.  This error is deceving and it has nothing to do with repomd.xml.  This has to do with the ftp address in your repo being wrong.

For instance:

Using ftp the path should be:

ftp://x.x.x.x/pub/...../.../../... (anything after pub can be whatever you created)

The key here is don't forget to pub!

Additional info: Link

Monday, September 28, 2015

Control+Alt+Delete on a Mac when RDP'ing to a Windows Machine

I had just found this and can't believe I never knew this. 


It works just the same as Control+Alt+Delete

Source:  Link

Thursday, September 17, 2015

Unable to unlock iPad/iPhone

At IOS lock screen, you can't swipe.  You have a block around the "Swipe to Unlock".  It is also talking every time you do something.

You have the Voice Over enabled.  Now this is a huge pain if you just reset the ipad and it is in setup mode.

To turn it off press the home button 3 times.  Try swiping again. It should be back to normal.

Source:  Link