Translate

Thursday, August 28, 2014

Change bash colors

Blue(34) as directory color is difficult to read and strain your eyes.
So, i googled and find using environment variable LS_COLORS one can modify it.

Here is one such google search result:

----------
These three lines in my .bashrc file allow me to configure the color system
by extension.

alias ls="ls --color=auto"
LS_COLORS="di=31;1:ln=36;1:ex=31;1:*~=31;1:*.html=31;1:*.shtml=37;1"
export LS_COLORS
And the output of this command:
dircolors --print-database | less
tells me about colors and codes etc that I can use..
--------

To change only directory color do this:
$ export LS_COLORS="di=31;1:

31 is for red
32 is for green
33 is for yellow
34 is for default blue

Authentication using ssh public (pub) and private keys(pem)

To avoid the need of supplying username and password everytime from trusted machine to login into your server we can generate pub/private keys to drop this authentication step from foreground

Steps involved to generate one such key pair are:
  1. Generate key pair (.pub and .pem)
  2. Pass .pub file to your server to store it in its authorized_keys file
  3. Keep .pem(private key) at yourself whenever to be used to login to server.

1) Generating key pair

     ssh-keygen -t rsa -b 2048 -v

It'll  generate 2,048 bit RSA key using verbose (questions asked during) mode, and a public .pem X.509 certificate.
Supply what it ask :

Generating public/private rsa key pair.
Enter file in which to save the key (/home/anonymouse/.ssh/id_rsa): hetzner
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in hetzner.
Your public key has been saved in hetzner.pub.
The key fingerprint is:
bb:c6:9c:ee:6b:c0:67:58:b2:bb:4b:44:72:d3:cc:a5 localhost@localhost
The key's randomart image is:

It wil generate two files at the specified location( default at ~/.ssh/) . File woul dbe named id_rsa.pub and id_rsa (if no name is supplied). Rename id_rsa ,the file without extension, to it_rsa.pem. This will be your private key.

Now don't forget to add the key to the ssh agent
      ssh-add keyName.pem
(Note: Do verify your ssh-agent is running.If not run it: eval `ssh-agent -s` )

2) Pass public key to your server to make this key pair work

   ssh-copy-id -i ~/id_rsa.pub root@your.server.ip

Follow the steps you see in the output of this command

Verfiy that you've all trusted keys listed in authorized_keys file on server:

   sudo nano ~/.ssh/authorized_keys or $ sudo cat ~/.ssh/authorized_keys

2) Test the connection now. Try login from client using private key(.pem) into your server:

  sudo ssh -i ~/id_rsa.pem root@your.server.ip


If you have multiple servers and prividing private key in the input is tedious for you, then generate ssh config file. This way you can access your server simply by SSH'ng into it by their name

SSH config 


  • Generate  ~/.ssh/config file with following content/template: 

Host server1 server1.company.com
Hostname 12.34.56.78
User ubuntu
IdentityFile /media/11361B1123123634/server1.pem
Host server2 server2.company.com
Hostname server2.company.com
User root
IdentityFile /media/11361B1123123634/server2.pem

Host myPC myPC.localHostname 192.168.0.106
User mike
IdentityFile /home/mike/.ssh/id_rsa



This file is recognized by ssh and would be used by other utilities like rsync as well.

Try SSHin'g now

  ssh server1

Troubleshooting:

  1.    Permissions on clients ~/.ssh should be dr-xr-x---
            chmod 550 .ssh
  2. Troubles with key path, rsync prompting for password when should not
    If using rsync with sudo, it looks for key file in /root/.ssh/config not in /home/user/.ssh/config, so be sure to copy or link this file to correct location, otherwise ssh and scp will be working fine while rsync will prompt for password.
  3. Error while running ssh-add
    vagrant@vagrant-ubuntu-precise-64:~$ ssh-add  ~/.ssh/id_rsa.pub
    Could not open a connection to your authentication agent.

    Here You might need to start ssh-agent before you run the ssh-add command:

    eval `ssh-agent -s`
    $ ssh-add
         if in root its' not working then try this:
            $    exec ssh-agent bash

---------------------
Reference:
    http://www.beginninglinux.com/home/server-administration/openssh-keys-certificates-authentication-pem-pub-crt

Getting started with Chef-Server and Chef-client

Steps Involved:

  1. Chef-server installation and key generation
  2. Launching chef-server webui.
  3. Transferring keys to admin workstation (admin.pem and chef-validator.pem)
  4. Adding new node 
  5. Deploying cookbook on newly added node (bootstrap)

1) Installation of chef-server
  • Download chef-server from this link: http://www.getchef.com/chef/install/


  • Create /etc/chef-server/chef-server.rb file 
    •  Use this file to configure it 
      • http://docs.opscode.com/config_rb_chef_server.html
  • Reconfigure it
    • $ sudo chef-server-ctl reconfigure
  • Test to verify installation
    • $ sudo chef-server-ctl test
      • Make sure your all test passed else, debug them.
  • Run all service
    • $ sudo chef-server-ctl start
  • Launch web-ui
    • https://ip.addres.of.yourMachine
      (in case of error, see the troubleshooting section)
      • If all service are running you'll be able to access web-ui through https protocol
      • Use default chef-server password which you can see at the right side on the login screen



  • Login and change password. Make sure to copy the private key and it's what our admin machine/workstation will use to connect to chef-server
  • Copy and name the private key to : "admin.pem"
2) Installation of chef-client on admin workstation
  • Make sure you've copied all  private keys(.pem files) from chef server to your admin machine.
    • These files are chef-webui.pem, admin.pem(which you recreated after login to webui), and chef-validator.pem
  • Create chef-repo ,preferrably inside admin home directory.
         I am creating it  under /root/chef-repo
    • git clone https://github.com/opscode/chef-repo
      • This is blank repo provided by opscode which we can use
    • Create .chef directory under chef-repo folder and copy all your  keys( STEP 1) in here.
  • Generate knife.rb file
    • cd chef-repo/.chef
    • Run command to let make knife generate knife.rb file or you can write  your own. Here we're taking aid of knife command.
      • $ sudo knife configure init
      • Enter what it ask you. After completion my knife.rb looks like this
        • log_level                :info
        • log_location             STDOUT
        • node_name                'admin'
        • client_key               '/root/chef-repo/.chef/admin.pem'
        • validation_client_name   'chef-validator'
        • validation_key           '/root/chef-repo/.chef/chef-validator.pem'
        • chef_server_url          'https://192.168.50.40'
        • syntax_check_cache_path  '/root/chef-repo/.chef/syntax_check_cache'
  • Try connecting to chef-server
    • $ knife client list
    • If you succeed connecting to server you would see following list:
      • chef-validator
      • chef-webui
  • That cover your chef-client installation: The admin workstation setup
3) Add new node and try boostrapping it with a cookbook
  • knife bootstrap is command to add new node into infrastructure
    • $ knife bootstrap IP.Addres --sudo  -x SSH_USERNAME -P SSH_PASSWORD -N NODE_NAME_TO_ASSIGN
    • eg
      knife bootstrap 192.168.50.43 --sudo -x vagrant -P vagrant -N slave03
  • List your new node via "knife node" command
    • $ knife node list
       slave03
  • You just added new node into your infrastructure. From webgui you  also see your new node listed under "node"  and "client" tab with no recipes currently added.
3) Writing and adding cooking for our nodes
  • Create new cookbook
    • $knife cookbook create apache
      • It will create a apache folder under ../cookbooks dir
    • Write recipe inside apache/recipes/default.rb  file
  • There are two ways of running recipe at client
    • SSH's node and run sudo chef-client
      • It'll pull recipe list assigned to it from server and run it
    • run bootstrap command from admin workstation but this time assign recipe list as well
      • $ knife bootstrap 192.168.50.43 --sudo -x vagrant -P vagrant  -r "recipe_01, recipe_02, ..."



    -----------------------------------------------------
    Troubleshooting
    1. lost private keys
      1. Lost of admin.pem file
        1. Visit webui 
          1. Edit account >> Edit >> Regnerate private key
        2. Lost chef-validator private key
      2. cleanse chef-server and start from scratch
        1. sudo chef-server-ctl cleanse
    2. ERROR: TypeError: can't convert nil into String
      1. Most probably argument error. I get this while executing following cookbook command:
          $ knife cookbook create apache
        • Reason it give me this error was my knife.rb file was incomplete. I didn't specify cookbook_path variable. This mean i've to explicitly specify cookbook path while executing knife commands.
        • I solved this by adding "cookbook_path" variable in my knife.rb file
          OR
        • giving -o argument to specify cookbook directory
          $ knife cookbook create MYCOOKBOOK -o /path/to/my/cookbook_dir
    3. Not able to communicate with chef-server
      1. /opt/chef/embedded/lib/ruby/1.9.1/net/http.rb:763:in `initialize': Connection re
      2.  
      3. used - connect(2) (Errno::ECONNREFUSED)
      • Most probably chef-server is not configured with right parameters like server name
      • Visit chef-server and cd /etc/chef-server  or wherever your chef-server is installed
      • make chef-server.rb file and enter following parameters

        server_name = "192.168.56.11"
        api_fqdn server_name
        nginx['url'] = "https://#{server_name}"
        nginx['server_name'] = server_name
        lb['fqdn'] = server_name
        bookshelf['vip'] = server_name
      • verify chef-server configuration :
        $ sudo chef-server-ctl show-config
      • More info here: http://stackoverflow.com/questions/19586040/install-chef-server-11-on-ec2-instance
    4. Old recipes are getting run on node 
    1. Most probably you forget to commit your changes and upload on chef-server
      1. $ knife cookbook upload cookbook_02
      Reference:
      1. http://www.getchef.com/blog/2013/03/11/chef-11-server-up-and-running/
      2. chef_server.rb and chef-server configuratioin:
        1. https://github.com/opscode-cookbooks/chef-server
        2. http://docs.opscode.com/config_rb_chef_server.html
      3. http://docs.opscode.com/config_rb_knife.html
      4. http://leopard.in.ua/2013/02/17/chef-server-getting-started-part-1/
      5. http://sanketdangi.com/post/50649257357/chef-11-configuration-aws-ec2-rhel-6-3-instance

      Understand Chef-Server Components (Erchef, bookshelf, webui, nginx-load balanceer, rabbitMQ, postregSQL)

      Chef-server:


      1. The server acts as a hub for configuration data
      2. The server stores cookbooks, the policies that are applied to nodes, and metadata that describes each registered node that is being managed by the chef-client
      3. front-end is written in erlang and hence the name Erchef




      ComponentDescription
      Bookshelf
      Bookshelf is used to store cookbook content—files, templates, and so on—that have been uploaded to the server as part of a cookbook version. Cookbook content is stored by content checksum. If two different cookbooks or different versions of the same cookbook include the same file or template, Bookshelf will store that file only once. The cookbook content managed by Bookshelf is stored in flat files and is separated from the server and search index repositories.
      All cookbooks are stored in a dedicated repository.
      WebUIchef-server-webui is a Ruby on Rails 3.0 application that hosts the web interface for the server.
      Erchef
      Erchef is a complete rewrite of the core API for the server, which allows it to be faster and more scalable than previous versions. The API itself is still compatible with the original Ruby-based server, which means that cookbooks and recipes that were authored for the Ruby-based server will continue to work on the Erlang-based server. The chef-client is still written in Ruby.
      Note
      Even though Chef 11.x is authored in Erlang, writing code in Erlang is NOT a requirement for using Chef 11.x.
      Message Queues
      Messages are sent to the Search Index using the following components:
      1. RabbitMQ is used as the message queue for the server. All items that will be added to the search index repository are first added to a queue.
      2. chef-expander is used to pull messages from the RabbitMQ queue, process them into the required format, and then post them to chef-solr for indexing.
      3. chef-solr wraps Apache Solr and exposes its REST API for indexing and search.
      All messages are added to a dedicated search index repository.
      NginxNginx is an open-source HTTP and reverse proxy server that is used as the front-end load balancer for the server. All requests to the Chef Server API are routed through Nginx.
      PostgreSQLPostgreSQL is the data storage repository for the server.



      References:

      1. http://docs.opscode.com/server_components.html

      Virtualbox : launching virtual machines with different IP's with network connectivity

      Virtualbox : launching virtual machines with different IP's with network connectivity

      1. Create instances as many you like but for all do following changes in the network settings
        1. Enable Two Adapters : 
          1. Set first one to NAT : To allow internet from host (eth0)
          2. Second to Host Only:  to allow different ip for every vm(eth1)
      2. Test instance
        1. fire 'ifconfig' command not down ip corresponding to eth1. All vm's will have different eth1 ip. Take note of them and use them to communicate with each other.
        2. Lauch them and verify that you're able to ping these vm's through 'eth1' IP