formtastic-bootstrap with Rails 3.2 and Twitter Bootstrap 2 2
Introduction
The Ruby world moves at an astounding pace. Pat Shaughnessy wrote an excellent series of articles in December 2011 documenting the options avaliable for using Twitter’s Bootstrap framework version 1.3 with Rails 3.1. At the time of writing Bootstrap has moved onto version 2.0, Rails is on 3.2.1 and Pat’s example application no longer builds as described.
A Slight Digression
In the rest of this post I’ll explain what little needs to be done if you’d like to follow those articles but use Rails 3.2 and Bootstrap 2.0, but first a quick digression on Bootstrap.
For the visually inept and graphically challenged amongst us, a set of professional and consistent design elements is a God-send. I’ve been using Perl (CGI.pm through to the later frameworks) and Rails since version 1.x to generate front-ends and dashboards and the like for all sorts of Infrastructure and traditional sysadmin tasks.
It just so happens that the Devops world follows, in part, the same route: assuming that system administrators can develop something other than shell script splattered with global variables, adopting Ruby the language from which the most prominent tools are built, and absorbing a huge amount from the Rails world: be it RESTful web services, rapid development or DRY. So, it is nice to finally be able to produce tools that look nice if for no other reason than quite often some fantastic operational work is trivialized and missed as it’s fronted by a bunch of crap in cgi-bin spitting out table elements.
Digression over, here’s what you need to do.
formtastic-bootstrap
cgunther has kindly done the hard work in getting mjbellantoni’s formtastic-bootstrap working with Bootstrap 2.0 in his bootstrap2-rails3-2-formtastic-2-1 branch. However, it requires the 2.1 version of formtastic, which is still in rc at the time of writing. However, the following in your Gemfile should do it:
gem 'formtastic', :git => 'git://github.com/justinfrench/formtastic.git', :branch => '2.1-stable' gem 'formtastic-bootstrap', :git => 'https://github.com/cgunther/formtastic-bootstrap.git', :branch => 'bootstrap2-rails3-2-formtastic-2-1'
Also, when editing your ./app/views/layouts/application.html.erb you should use the new Bootstrap 2.0 classes:
<body>
<div class="navbar navbar-fixed-top">
<div class="navbar-inner">
<div class="container">
<a class="brand" href="#">OrigamiHub</a>
<div class="nav-collapse">
<%= tabs %>
</div>
</div>
</div>
</div>
<div class="container">
<%= yield %>
</div>
</body>
Finally, you should heed the formtastic deprecation options, and construct your semantic forms thus:
<%= semantic_form_for @widget do |f| %>
<%= f.semantic_errors %>
<%= f.inputs do %>
<%= f.input :name, :hint => "The wangdoodle is a best-seller" %>
<%= f.input :type, :hint => "We only do three sizes!" %>
<% end %>
<%= f.actions do %>
<%= f.action(:submit) %>
<% end %>
<% end %>
tabulous
You’ll need to use the:
config.tabs_ul_class = "nav nav-pills"
option within tabulous to get the navigation bar to behave properly, along with the options recommended in the original article.
Conclusion
I’m sure the work above will be merged back into the mainline for each of the respective gems, and it’s a tribute to the community of github that such a lot of good work is given and fixed freely.
From the vault: "A script to monitor log files and add persistent offenders to /etc/hosts.deny"
I’ve been sorting through some old code, and apparently on the 10th of September 2008 at 08:49 I felt compelled to write a daemon in Perl that would add persistently connecting source IPs to hosts.deny if they continually abused sshd.
I remember doing this: I was between jobs and somebody somewhere whom I’ve long abandoned to my mail archives asked for it, and I had nothing better to do. So, for the sake of posterity, here is probably the last piece of significant Perl I ever wrote before making the move to Ruby - make of it what you will:
#!/usr/bin/perl -w
#
# ssh-deny.pl - A script to monitor log files and add persistent offenders to /etc/hosts.deny
#
# Author: Sam Pointer
# Contact: sam@outsidethe.net
# Version: 0.03
#
# Usage:
#
# Should a given IP address connect more than the specified number of times, add
# it to the TCP wrappers host.deny file.
#
# Note that this script simply parses the ssh_log file for the number of failures
# for a given IP address, tests that against a threshold, and adds a tcp wrappers
# rule if that threshold is exceeded. Therefore, if your log files roll around
# daily, more than $max_connections failure in that 24 hour period will cause a
# rule to be generated.
#
# It will look for and add a rule in the format:
#
# sshd : 66.6.136.62 : deny
#
# See the hosts_access manpages or TCP Wrappers documentation for more information.
#
# Generally the script should be invoked as UID 0 (root user), due to the permissions
# set on the hosts.deny and log files to be scanned.
#
# The configuration option @exception_list contains a list of full or partial IP
# addresses that will never be blocked. See the example configuration for the format.
#
# When started the script will detach from the console and become a daemon. It can
# be terminated via a SIGHUP/signal 15.
#
# DISCLAIMER: USE THIS SCRIPT AT YOUR OWN RISK. NO WARRANTY IMPLIED OR GIVEN. TEST
# ON A NON-PRODUCTION BOX FIRST.
#
# I've tested this on my own machine and it works fine. Change the configuration below
# to some files you can offord to loose first. Only $deny_file is opened for writing,
# so to test I copied that to my home directory, set the path there and checked that
# the rules added were correct, without affecting my live /etc/hosts.deny file.
# -- Configuration ----------------------------
my $max_connections = '3'; # Maximum number of denied connections
my $failure_string = 'Failed password'; # Always present on a failed connection
my $ssh_log = '/var/log/secure'; # Log file to scan for failed connections
my $deny_file = '/home/hosts.deny'; # Location of TCP Wrappers host.deny file
my $daemon_list = 'ALL'; # daemon_list string to add to hosts.deny. See hosts_access(5)
my $sleep_period = '30'; # Value in seconds to sleep before parsing log again. Adjust to suit system load.
# A list of full or partial IP addresses to never block
my @exception_list = ('192.168', '81.214.108.250');
# ---------------------------------------------
# Internal Global Variables
my ($record);
my (%failed, %blocked);
my $ip_regex = '\b(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b';
# Perlisms
use strict;
use POSIX qw(setsid); # Required to daemonize the script
# Make ourselves into a daemon
daemonize();
# Loop perpetually as a daemon
while(1) {
# Get a list of failed user IP addresses from the log file
%failed = get_failed($ssh_log);
# Get a list of what's already blocked
%blocked = get_already_blocked($deny_file);
# Move through the keys of our failure hash. Anything with
# more than $max_connections should be written to the file,
# providing it is not already blocked by a rule and is not in our exceptions list.
open (DENYFILE, ">>$deny_file") || die "ssh-deny: Cannot open $deny_file for writing\n";
foreach $record (keys(%failed)) {
if ( ! $blocked{$record} && $failed{$record} > $max_connections) {
print DENYFILE "$daemon_list : $record \n";
}
}
close DENYFILE;
sleep $sleep_period;
}
# -- Subroutines ------------------------------
sub get_already_blocked {
# This subroutine will parse $deny_file looking for all IPs
# in a rule that matches $daemon_list. These are returned as
# a hash for matching against later on
# Local Variables
my ($deny, $record);
my (@fields);
my (%already_denied);
# We expect a deny file to be passed. Open or die.
$deny = pop(@_);
open (DENY, "$deny") || die "ssh-deny.pl: Cannot open $deny for reading\n";
# Move through the file. For any rule that matches $daemon_list get the IPs
while ($record = ) {
@fields = split(/ /, $record);
if ($fields[0] eq $daemon_list &&
$fields[2] =~ $ip_regex) {
$already_denied{$fields[2]}++;
}
}
close DENY;
return(%already_denied);
}
sub get_failed {
# This subroutine retrieves a list of failed logins and returns a hash of IPs, times, and
# failed connections.
# Local Variable declarations
my ($log, $record, $marked, $exception);
my (@failure_records, @fields);
my (%failure_stats);
# We expect a path to be passed. Open the file or fail.
$log = pop @_;
open (LOG, "$log") || die "ssh-deny.pl: Cannot open $log for reading\n";
# Iterate through the log file, selecting rows that have failed connections
while ($record = ) {
if ($record =~ $failure_string) {
push @failure_records, $record;
}
}
# Close the log file
close LOG;
# Build a hash for each IP that has a connection
while ($record = pop(@failure_records)) {
@fields = split(/ /,$record); # Field index 13 is the IP address
$failure_stats{$fields[13]}++; # Increase failure counter for IP
}
# If any of the failed connections are in our @exception_list (never block)
# ensure that the IP is deleted from the hash so that they aren't blocked.
foreach $exception (@exception_list) {
foreach $marked (keys(%failure_stats)) {
if ($marked =~ /$exception/) {
delete $failure_stats{$marked};
}
}
}
# Return our hash with IP addresses as keys, and counts for each IP address as values
return(%failure_stats);
}
sub daemonize {
# This subroutine handles detaching the console, forking a new process, etc.
chdir('/') || die "ssh-deny: Cannot chdir to /\n";
open (STDIN, '/dev/null') || die "ssh-deny: Cannot read /dev/null\n";
# Uncomment the next two lines if you really don't want this to
# echo anything out as a daemon. Probably best to leave it
# so any error messages make it to the console.
#open (STDOUT,'>>/dev/null') || die "ssh-deny: Cannot write to /dev/null\n";
#open (STDERR,'>>/dev/null') || die "ssh-deny: Cannot write to /dev/null\n";
defined(my $pid = fork) || die "ssh-deny: Cannot fork\n";
exit if $pid;
setsid || die "ssh-deny: Cannot start a new session\n";
umask 0;
}
nf_conntrack: table full - how the absence of rules can lead to unexpected behaviour
I recently observed the dreaded:
nf_conntrack: table full, dropping packet
message on a host that formed part of the external tier of an infrastructure, where we expected, managed and throttled many connections. The odd thing was, the hosts should have been doing nothing iptables-wise to be tracking connections or otherwise generating this message. On behaving and misbehaving hosts both an `iptables -L` would show a bunch of empty chains. Odd.
However, a few leaps of logic later lead to the following being discovered on the well-behaved hosts:
# lsmod | egrep 'ip_tables|conntrack' ip_tables 9899 1 iptable_filter x_tables 14175 1 ip_tables
and curiously this on the mis-behaving hosts:
# lsmod | egrep 'ip_tables|conntrack' nf_conntrack_ipv4 10346 3 iptable_nat,nf_nat nf_conntrack 60975 4 ipt_MASQUERADE,iptable_nat,nf_nat,nf_conntrack_ipv4 nf_defrag_ipv4 1073 1 nf_conntrack_ipv4 ip_tables 9899 2 iptable_nat,iptable_filter x_tables 14175 3 ipt_MASQUERADE,iptable_nat,ip_tables
Sure enough, we can see why nf_conntrack is now involved in the TCP stack and why we might be filling up its buffers, but it doesn’t explain the disparity between the hosts.
In retrospect the explanation is both blindingly obvious, craftily subtle and provable to boot. In short, when a rule is added to the ‘nat’ iptables table the various kernel modules required to support it are dynamically loaded. They remain, and are therefore part of the execution path of iptables, even if their contents is flushed. What this means in practice is that for a running kernel, once you have defined a nat iptables rule you are at the mercy of its buffer size and other constraints for the lifetime of that kernel run. Or put more simply, creating and flushing nat rules does not leave you in the same state as having never created them.
We can prove this in a rather ham-fisted way.
We’ll create a small dummy client and server in Ruby for the purposes of opening many concurrent connections. We’ll manipulate some of the limits down in order to enable us to re-produce the error without requiring massive live scale. The following scripts are best run under Ruby 1.9 so that we can make use of native threads.
#!/usr/bin/env ruby1.9
#
# Accept many connections
#
require 'socket'
server = TCPServer.open(7777)
loop {
Thread.start(server.accept) do |client|
loop {
sleep 60 # do nothing
}
end
}
#!/usr/bin/env ruby1.9
#
# Connect many times
#
require 'socket'
host = 'localhost'
port = 7777
19998.times do
Thread.start do
TCPSocket.open(host, port)
loop {
sleep 60 # do nothing
}
end
end
As root, we’ll run the following to open up our ability to create connections:
ulimit -n 20000 echo 20000 > /proc/sys/kernel/threads-max echo 0 > /proc/sys/net/ipv4/tcp_syncookies iptables -L # forces the 'ip_tables' kernel modules to be loaded with empty tables and chains
If you then run ./server.rb followed by ./client.rb and `watch -n 2 “dmesg | tail -10”` you’ll see, well, not much going on. However, if we introduce and then flush and delete a nat table iptables ruleset we’ll see both the modules loaded and the tests produce the expected error in the kernel ring buffer output:
iptables --table nat --append POSTROUTING --out-interface eth0 -j MASQUERADE iptables --flush iptables --table nat --flush iptables --delete-chain iptables --table nat --delete-chain ... # lsmod | egrep 'ip_tables|conn' nf_conntrack_ipv4 10346 3 iptable_nat,nf_nat nf_conntrack 60975 4 ipt_MASQUERADE,iptable_nat,nf_nat,nf_conntrack_ipv4 nf_defrag_ipv4 1073 1 nf_conntrack_ipv4 ip_tables 9899 2 iptable_nat,iptable_filter x_tables 14175 3 ipt_MASQUERADE,iptable_nat,ip_tables ... sysctl net.netfilter.nf_conntrack_max=100
If we run the same tests again with the artificially low limit and monitor the kernel ring buffer with `watch -n 2 “dmesg | tail -10”`once again you’ll quickly see the “nf_conntrack: table full, dropping packet” message.
So, what have we learnt here? The short of it is that manipulating nat tables under iptables on a running kernel will change the behaviour of your network stack, and that clearing down any nat tables will not return the stack to the same previous state. In order to do that you’ll have to:
rmmod iptable_nat rmmod ipt_MASQUERADE rmmod nf_nat rmmod nf_conntrack_ipv4 rmmod nf_conntrack rmmod nf_defrag_ipv4 ... # lsmod | egrep 'ip_tables|conn' ip_tables 9899 1 iptable_filter x_tables 14175 1 ip_tables
to return things to the previous state, at least for this example. Whether that is preferable on a production world-facing system to a reboot or a recommission is open to debate.
EDIT:
Some further testing on my part has determined that even listing the nat tables with `iptables -t nat -L` will cause the conntrack modules to be probed into the kernel. For very busy world-facing hosts the only solution that I can see is to add the various conntrack modules to /etc/modprobe.d/blacklist.conf to ensure that they are never loaded.
I’ve seen this on CentOS/RHEL and Ubuntu, right up to the current server release of the latter.
Installing Nagios on Ubuntu or Debian without Postfix
If you install the default ‘nagios3’ package from the repositories on a Debian-based distribution, you wind up with a full copy of postfix installed. This is fine if you’re simply trying to get the thing to work, but as part of a wider infrastructure you most likely do not want a full-fledged MTA arbitrarily popping up on your Nagios host - an MTA that you have to administer, monitor (!), patch and most importantly secure.
The dependency chain that causes postfix to be installed is:
nagios3 → nagios3-core → nagios3-common → bsd-mailx → default-mta | mail-transport-agent.
Why the package maintainers made bsd-mailx dependent on a fully-fledged MTA I will never know. Perhaps they wanted to ensure things “just worked”? It still seems a bit heavy handed to me, especially when one can configure .mailrc to point to a mailhost and be done with it.
In order to install nagios3 from the repositories and satisfy those dependencies without pulling in postfix you should install the ‘lsb-invalid-mta’ package, which provides ‘mail-transport-agent’ and satisfies the dependency chain above, in place of postfix. The package provides a sendmail binary that does nothing but return a non-zero return code, so you’ll never accidentally send mail from a local system, but you will have to configure your system to take advantage of a suitable MTA host.
Here is some puppet to install nagios3 without postfix:
# /etc/puppet/modules/nagios-server/manifests/init.pp
#
# Class: nagios-server
#
# This class maintains a Nagios server.
#
# Parameters:
# None
#
# Requires:
# nagios-server::install
#
class nagios-server {
include nagios-server::install
service { 'apache2':
ensure => running,
enable => true,
require => Class['nagios-server::install'],
}
service { 'nagios3':
ensure => running,
enable => true,
require => Class['nagios-server::install'],
}
}
# /etc/puppet/modules/nagios-server/manifests/install.pp
#
# Class: nagios-server::install
#
# This class will install a Nagios server from the repo packages
#
# Parameters:
# None
#
# Requires:
# Nothing
#
class nagios-server::install {
# Prevent nagios3-common->mailx dependency from pulling in an MTA.
package { 'lsb-invalid-mta':
ensure => present,
}
$packages = ['nagios3', 'nagios-images', 'nagios-plugins', 'nagios3-doc',]
package { $packages:
ensure => present,
require => Package['lsb-invalid-mta'],
}
}
