Archive | PostgreSQL

The free database server available at http://www.postgres.org

HowTo notes: installing REE, Passenger, Ruby on Rails on CentOS5

Recently, I installed Redmine, the Rails-based project tracker,  on an existing virtual private server, running CentOS 5 Linux and an assortment of LAMP applications and web sites using Apache, PHP and MySQL. I chose Ruby Enterprise Edition (REE), an optimized and high-performance version of Ruby, and Passenger, an Apache module to connect to Ruby. Here are the steps I followed, as a reference and in hopes it might help others:

  1. Log onto the box via ssh. As a good practice, I run as an unprivileged user, and use sudo only when superuser rights are needed. So, download the files and un-tar them and read the READMEs and run the processes as a regular user, switching to root via sudo only when the rights are needed.
  2. Download and install Ruby Enterprise Edition. Today, that’s:
    wget http://rubyforge.org/frs/download.php/68719/ruby-enterprise-1.8.7-2010.01.tar.gz

    but follow the instructions you’ll find at http://www.rubyenterpriseedition.com/download.html for the latest. Those instructions will have you un-tar the file and then run the installer.

  3. Install the tools needed to build Ruby Enterprise Edition. If you run the installer, it will prompt you for missing tools. In my case, I needed to:
    sudo yum install gcc-c++ make patch zlib-devel openssl-devel readline-devel
  4. If you’ll be using mysql, you’ll need the native code to build the matching gem. REE attempts to build gems for MySQL, SQLite and PostgreSQL, so choose your flavor and issue the appropriate command like:
    sudo yum install mysql-devel
    

    or postgresql-devel or sqlite-devel as appropriate.

  5. Follow the instructions supplied by the REE installer. They did a nice job on that. To install Passenger when I was done, some additional source code files were required:
    sudo yum install httpd-devel apr-devel
    sudo /opt/ruby-enterprise-1.8.7-2010.01/bin/passenger-install-apache2-module
  6. Once again, follow the prompts to add the proper load module and path commands to your web server configuration, add a configuration file for your site, and you’re ready to start installing your site’s code.

NOTE: It wasn’t the case with my particular host, but you may need to adjust your SELinux settings, if you’ve got SELinux enabled. See the post at http://www.catapult-creative.com/2009/02/04/installing-rails-on-centos-5/ and look for “SELinux” for some ideas.

Notes from Python Special Interest Group, 20-Nov-2009

Eight people attended the Python Special Interest Group, held a week early to avoid the Thanksgiving holiday. Anticipate a reschedule December meeting as well.

Last night’s meeting was a vigorous and far-reaching discussion of MySQL, Oracle, the future of MySQL, Maria DB, OpenOffice.org automation using Python, OpenOffice.org automation using Visual FoxPro, Twisted, IE6, Zope, Plone, Django, MS SQL Server, pyodbc, SQLAlchemy, Cascading Style Sheets, IE6, FireFox and FireBug, User Agents, IE6, how not to insulate a bungalow roof, the (Python!) cssparse module (http://cthedot.de/cssutils/), Fortune’s selection of Steve Jobs as “CEO  of the Decade”, Lenovo netbooks and Ubuntu, the Millennium, why calendar years are one-based and not zero-based, distributed version control systems, master-slave and master-master replication using MySQL and Postgres, svn and git, and more! Whew! You should have been there!

Thanks to Bill for organizing the meeting, to all for attending and participating, and to the Amoskeag Business Incubator for providing the great facilities!

Stay tuned for an announcement of the December meeting, and hope everyone has a good Thanksgiving!

Notes from PySIG, 28-May-2009

It was a dark and stormy night. Nonetheless, six members made it to the May meeting of the Python Special Interest Group, held as usual on the fourth Thursday of the month at the Amoskeag Business Incubator in Manchester.

We had an Open Mike Night format, a round-table discussion where everyone shared what they were working on.

I plugged upcoming meetings, available as always at http://gnhlug.org — MonadLUG in particular, is to be praised for posting 4 months worth of meetings in advance.

Mark has a client who’s weaning off a proprietary OS and looking for a replacement document management system / word processing system, and is considering LyX, which is a  front end to LaTeX and has numerous utility scripts written in Python. Mark asked for suggestions for additional resources and the two Bills were able to come up with some ideas.

Arc talked about some wireless technologies he’s researching (neat stuff!). Arc also reported the Gaming SIG is coming along nicely: 5 people at the first meeting, 10 at the second. Details at gnhlug.org . Hoping to schedule a FPS (First Person Shooter) night soon. Coming up next Friday June 5th, the SIG will take a look at the awesome audio utility, Audacity, as it relates to gaming, and then engage in the Battle for Wesnoth.  Gaming SIG meets at the Brady Sullivan building in the DynInc offices on the fifth floor – see http://wiki.gnhlug.org/twiki2/bin/view/Www/GamingSIG.

Shawn O’Shea completed a course in Network Design and Planning at UMass Lowell (and got an ‘A’, congrats!) and showed us his lab work, written in Python! He very bravely showed us his code and we talked about some of his algorithms and looked at a couple of the modules he used, including optparse, netaddr and cmd.

Bill Freeman reported he’d been working in Plone and Python 2.4 and missed some of the features available in later versions. He created some code to address the worst of the deficiencies, and hopes to be able to release it freely soon. Stay tuned.

Thanks to Bill for organizing the meeting, to the Amoskeag Business Incubator for the fine facilities, to Arc for bailing us out with an extension cord, to Janet for the awesome (!) cookies, and to all for attending and participating!

Voices That Matter, Professional Ruby Conf ’08 Boston, Day Two

Matt Knox did a stand-up presentation for the second day keynote, as Giles Bowkett was not able to make it to the conference. Matt was telling us about the great stuff he’s learned at conferences. Smalltalk is the coolest language ever. Unit Testing is not the greatest thing ever. Damien Katz’ great story of success. DHH wrote some framework, we’re not the fringe any more. The Great Surplus (I believe this talk is available on IT Conversations, search for RailsConf). Giles, in a picture, bowling and archery at the same time. Giles (in words): an epic tale of adventure, fail, music, rebar, LSD, VC, parking-ticket evasion, and pet monkeys (that’s be us).

Followed an informal Smack-Down session: one yellow sheet, one blue sheet, Obie Fernandez pops up slides and we get to vote, vim vs. emacs, Agile Dev w/ Rails vs. The Ruby Way. svn vs. git broke out into a debate, Brian argued that svn is for maintaining versions of files, not for collaboration, and is fundamentally broken and people need to get over it. jQuery vs. Prototype? Chad from Thoughtbot uses Prototype, and says currently the pain of switching outweighs the pain of sticking with Prototype. grid960 (http://960.gs/) versus… I missed it, Sinatra v. Camping? Few if any cards, all for Sinatra. Blueprint v. YUI, empty? v. blank?, system gems v, frozen gems, Vlad v. Capistraro, Mocha v. Flexmock, FactoryGirl v. ObjectDaddy, resource_controller v. make_resourceful, HAML v. ERB: interesting debate: HAML “Beautiful but astere”, ERB parses like HTML. HAML less typing & faster, ERB has too many close-tags, Obie argues that HAML maps well to CSS – I need to check this out!, Redcloth v. Bluecloth (no one), RMagick v. ImageScience (nobody’s sure) , Tiny MCE v. FCKEditor, restful_authentication v. clearance, Test::Unit v. RSpec (RSpec 3:1). Very informative!

Payment Processing with Active Merchant, Cody Fauser, Shopify
Why? Get money from your customers. Originally http://www.snowdevil.ca/ integrated with Moneris, a single payment processor. Expanding to support multiple processing options, everyone has their unique API, glossaries, , “unique snowflakes,” etc. Simple Uniform API to dozens and dozens of payment processor back ends in over a dozen countries. Components follow the normal flow of Customer data entry, Authorization, Payment and Settlement. Includes extensions for the Address Verification System and Card Security Code (CID, CVC or CW2). Cody jumps into the code. It’s simple. And it’s the same for three processors he shows — that’s the point, after all. The code for settlement is drop-dead simple as well, and for collecting the results.

Rock Solid Ruby Deployments, Philippe Hanrigou
Addison Wesley Shortcuts, Troubleshooting Ruby Processes, free chapter available on line at InformIT. How to troubleshoot problems with with threads. An http request gets the attention of the Rails handler, which gets the lock, goes to the dispatch, which might go to the database. While waiting on the I/O, a new request gets a new handler spawned which… goes to get the lock, which is busy, and… goes to sleep waiting. Ouch! This drove the need for caller_for_all_threads, which can be used to dump the state of all threads. This is a patch, not a gem, but will be included in next version of Ruby Enterprise Edition and Passenger. Another good tool for troubleshooting is DTrace “a comprehensive, dynamic troubleshooting tool” or in Brian-talk “a freaking awesome tool for debugging all the fracking stack.” DTrace runs within the kernel and so it can see all the hangups from the Ruby interpreter to the networking calls to the database processes to the raw I/O. However, DTrace is not running on Linux. “How many of you deploy on Solaris? Open Solaris OS X Server?” Not a single hand. “We all deploy in Linux.” A DTrace is not Linux-friendly. And SystemTap, while venerable is not DTrace. Paul Fox, www.crisp.demon.co.uk, is trying to port. Had a demo of DTrace running on Ubuntu in a VM. PID Provider is a wrapper that can intercept calls from an application that’s not instrumented for DTrace. Until then, we talk about what we can troubleshoot now. Ruby Probes, Joyent, good wy to get started. Ben Rockwood has a good presentation on MySQL and DTrace from the MySQL Conference 2008, find on the web. There’s also a ruby plug-in for DTrace: gem ruby-dtrace ‘Tracer”
http://github.com/ph7/xray/tree/master

Blaine Cook, formerly of Twitter
A short story about messages… things we do every day: postal mail, telephones, etc. Synchronous v. asychronous. Clustering, hadoop documentation, Google MapReduce. Reduce latency. Messaging: has a sender and a recipient. Synchronous: bad. Options: RabbitMQ, ActiveMQ, Spread, beanstalkd, Starling (Blaine wrote for Twitter), Gearman/TheSchwartz, Delayed::Job, cron. Each has their own protocols, some share a few like AMQP. Does queue need durability? What kind of guarantee can you get on delivery? Beanstalk doesn’t have a durability guaranty, and Spread only under certain configurations. Reliability? Atomicity? If a job is sent off into a queue and no success or failure message is received, how should the messaging system behave? Restart the job? What if the job has a cost (example: sending SMS) whose cost is incurred, but completion fails later? Recently released ActiveQueue, some similarities to Nanite. Others: Workling talks to Sterling, Rabbit, ActiveMQ, etc. Paraphrasing, if email worked the way the internet works now, you’d have to check individually to see if each of your friends had sent you email.

Matt Pelletier (CTO, Eastmedia) was to talk “Rails Scales” but instead talk about “Mobile Developments” taken a great interest in the mobile sphere, business has grown out of web-only focus. Global PC stats: 280 million PC sales/year. Phones sold: 1.3 BILLION, more in a single quarter than all computer sales. “Smart” phones have small share of that market, admittedly, but growing. Technologies: Voice, SMS, MMS, Mobile Data Services. Voice: regular calls, robo calls, automated voice services (airlines), VOIP, Voice XML, Asterisk/Adhearsion. SMS, finally exceeding voice in US, has been the situtation in Far East for a while: Mobile Terminated (MT), Mobile Originated (MO), Shortcode vs. Phone number, No friggin’ metadata, Project: Mobile Commons (launched 2006, worked with civic organizations). MMS: “Total Shit Show,” Carrier Specific, Event Apple said screw it, use email instead. What’s up with viewmymessages.com???? Mobile Data Services: WAP, the “Real” internet, Apps. WAP 1.0 complete disaster, WAP 2.0 is basically trimmed down Web. Project: New York Jets Mobile. Page caching rocks. Brett Farve signed, 4 million page views in a couple of hours, cached through Akamai. The “Real” Internet: as long as you exclude Flash, Real browsers with Js/AJAX support, marketing heating up, MobileSafari, Android Browser (WebKit based), Fennec (FireFox), Opera, Skyfire. Apps: used to be controlled by the carriers in the US, Qualcom’s BREW, Apple iPhone via App Store, Google Android offers Marketplace. App Technology: (most of these folks are too young to remember OS/2 v. Windows) too many OSes, entrenched. Interesting stats on numbers of smart phones sold (Apple #2 to Nokia), as of Q4 more messages that voice calls in US. What are people doing on their phones? Watching videos, accessing social sites, Matt said “boring but played out” but I’m not sure of the context. Up and coming: Geo-related apps: directions, find a restaurant. GPS is still really slow, AGPS (tower triangulation) much faster, less accurate, Skyhook Wireless FTW, “I want to know exacly where I am, in motion” – makes trouble playing games that are exacly location-dependent. New product, RSN, http://spot.io – “tell us what you saw and where you saw it.” iPhone development… feels like the Rails community, sort of: searching sparse domain knowledge; Objective-C ain’t Ruby; iPhone SDK ain’t Rails. “But STFU, it’s pretty badass for 6 months in.” A decent coder can get going… it’s fun and excitig. For Spot.io we wanted the photos off the camers and quick entry.

Jon “Lark” Larkowski, HashRocket, Communications to support Client Development
Pivotal Tracker: we don’t write a line of code that doesn’t support a user story. Current iteration gets its own page listing ongoing stories, amount of effort. Let’s clients view velocity and acceleration, where stories are in the process, etc. Stories need to be full (BAD: “admin interface”) so any dev can sit down and start coding. Direct capture to Tracker, slowly train customers to write their own. Moving toward standard forms, ideally, Cucumber forms As a l Pair Programming (ATFT). d2d: keep focused, honest, high quality. two wheels on the axle. Company-wide standup each morning: what did you do, what goes next what. Communication aspects: c2w express stores as test, d2c RSpec specdoc format, cucumberl d2d: durable system specification. Techniques used every day at HashRocket: RSpec, MVC test, Factory Girl Object Daddy, Cucumber. Selenium, RSpactor, Continuous Integration, Clicking on Stuff. Delivery: all the f’ing time (Sorry, Brian). Deliver finished stories multiple times a day, c2d very tight feedback look with client, then d2d back together (2 pairs programming separately meets to exchange what’s new). Engine Yard staging slice (highly recommended) or EC2 with Passenger. Funny diagram of “Impedance Mismatch” with client communication, dev communication,

David Berube, Surviving as a Windows-based Ruby developer
Windows is a difficult platform to develop Ruby on, since it lacks many of the POSIX tools (natively, there are add-ons) and running Apache is less well-known. David’s pretty frank: he recommended a Windows dev look seriously at Ubuntu (Disclaimer David was my predecessor as “Fearless Leader” of the Greater New Hampshire Linux User Group, and I think we’re both equally, correctly prejudiced). Intrinsic pitfalls: automatic binary conversions (LF->CRLF) can corrupt binaries. No Kernel#fork; this won’t work. No GNU out of the box. No compiler out of the box. There are some good options: Editor: Visual Studio: Ruby in Steel, Net Beans, gvim (recommended) cream, Emacs, “E Text Editor” – Textmate clone (bundles can be work, slashes are the wrong way, someone has a patch bundle for those, too) Cygwin, UNXtils (preferred). Misc tools: Safari for Windows, FireFox, Tortoise for SNC, LDE(x) windows desktop replacement, PuTTY, FileZilla, SmartFTP

Rails AntiPatterns, Chad Pytel and Tammer Saleh, Thoughtbot
Moving code from controllers to the models: ‘create’ method, 14 lines is too long. Smells funny: there’s a transaction that wraps a single call create_version!() Eliminate DRY code in create new version and create first version, use defaults to set default/initial values of version, use created_at and id functionality built into ActiveRecord design. Slide entitled “Minor Law of Demeter Violation” where they short-circuit the redundant code that references an object and drills into its properties.; better to expose that property/attribute/structure as an explicit method in one place and use the new method to free multiple dependencies on an internal structure. Simplified the create version method to half-dozen lines, removed a related trigger as a callback rather than explicit code. Lessons learned: even a 15 line controller action is too long. Exceptions should be exceptional, Your models should use callbacks to add complex behavior. 2nd example: way too much code in update. Solution: use a constant with a hash to create a set of methods. Lessions: tackle large refactors iteratively. push as much business logic into the model as possible. In lots of code, dimple DRY principles will lead to a lot of refactorings. Common problem domain: user and roles. “Thoughtless Code” Methods like Role, Roles, CanEdit, CanPublish, CanDoSomethingElse…. Solution: Remove the whole shebang, add boolean columns to User: Admin, Edit, Publish, and Active Record automatically gives you the elegant User.Edit?, User.Publish? User.Admin? Lessons: Refactoring! Bug Fixing, Always Be Refactoring, didn’t cover testing; there was a test suite, so regressions were avoided.

Battle Royale, Foy Savas, author “The Merb Way”
Merb’s Role in the MVC Holy Wars
Merb is Rails rewritten with Django and Pylons influences. Created by Ezra in 120 LOC Mongrel _ ERB. Now: high performance, quick to deploy, easy to scale. Sounds like Rails. It isl Awesome. Rails vs. Merb in a fight cage? Point by point comparison. “No code is faster than no code.” “A micro-second saved is a micro-second earned” Background processes, run_later block. Built on Rack Rack allows ou to string together apps using Rack::Cascade. Example, just serving RSS feeds don’t have to invoke entire Merb framework. ActiveSupport is a kitchen sink, ExLib isn’t. DataMapper destroys ActiveRecord. Authomigrations are what your developers always needed. Lazy Loading DataSets. Strategic Eager Loading. Plugins are Gems. Merb Parts. Arguably one of the slickest presentations at the conference, Foy’s rendering of the Dao De Jing, which he read in what I presume is the original dialect, “A way that can be taken rarely stays the way. A name that can be named rarely stays the name.” Not just dramatic, but the theme of his presentation. Perhaps Merb will be the next way.

Continuous Learning Curve: Javascript

I’ve avoided spending too much time delving into Javascript. My four-year switch from Windows-uber-alles (including VFP, VSS, SQL Server, Ingres, Oracle, HTML, OLE, ODBC, SCC, COM, XML, MCSE, MCSD, XSLT, DCOM, RSS, MS Office, Exchange, MAPI Bad, SMTP Good, MVP and more acronyms!) to Linux-Apache-MySQL-Postgres-PHP-Python-Ruby, not to mention XHTML, CSS, bash, Smarty, Django, TWiki, dojo, et al had kept me busy enough. But a new client assignment needs a highly-interactive web site and dropping in great big globs of someone else’s Javascript is not going to solve all the problems; at a minimum, I’ve got to be able to read it, debug it and tune it for the client’s particular needs.

Did you know that a limited version of Safari, the O’Reilly online library, is included with a membership to the Association of Computing Machinery? I’ve been an ACM member for years and been meaning to get around to trying this out. My Javascript studies seemed the perfect occasion. I’m reading Shelley Power’s Learning Javascript online and getting quite a bit out of it. I love when you settle down with a book and start going “Oh, is that what that meant?” or “Now I get it!”

State of the Computer Book Market, Part 4 – The Languages – OReilly Radar

Over at O’Reilly, Mike Hendrickson posts State of the Computer Book Market, Part 4 – The Languages – OReilly Radar. Bear in mind in the world of lies, damned lies and statistics, these are statistics. Like measurements of job posting vs. languages, or web pages vs. languages, these aren’t an indication of what languages are in use out there, or what languages are suitable for use. This is just a measurement of what book titles sold the most, grouped roughly by language. If the best book on your favorite language is years old, it’s probably shown as small here. If a bunch of crappy books got published on the language-of-the-week, and the authors and publishers did a good job of getting a buzz going, it’s probably scoring high. Don’t use this to decide whether to use vi or emacs, tabs or spaces.

But it’s interesting data.

Dabo rocks!

I’ve mentioned it before, but the dabo project rocks! dabo is intended to be a cross-platform (Mac/Linux/Windows/Everywhere) rich-client application (like FoxPro 2.5 before MS bought it) with the rich-client experience (grids, list boxes, checkboxes, pageframes, menus, multiple forms) in the appropriate widget-set for each OS. It supports a slew of backend data sources (MySQL, PostgreSQL, MSSQL, Oracle, more) and is designed with a similar architecture (UI-BizObjects-Data) to many of the FoxPro frameworks. Best of all, it’s written in Python and available under an Open Source license.

I’ve spent a couple days downloading the source, watching the excellent screencast tutorials, browing the extensive mailing list archives and wiki, running the demos, generating an app with the App Wizard and reading the code. I’ve got an existing LAMP application that would benefit from a rich-client component with reporting capabilities, and dabo looks like a good choice. Hope to blog my progress as I get into it.

Mad Penguin interviews PostGreSQL PR Lead

OSNews points to an Interview with Josh Berkus of PostgreSQL. “The PostgreSQL database project has recently released Version 8.0, which was received with quite some fanfare, mostly due to its first-ever Windows port. Mad Penguin talked with Josh Berkus, one of the core team members, to find out how 8.0 has fared since its official release on January 17, 2005.” Interesting and brief reading for those curious about the project. While MySQL has gotten the lion’s share of the attention in the Free/Open Source Software market so far, PostgreSQL is a worthy and feature-packed competitor.

Webmin releases version 1.180

A new version released today. Webmin is a great tool for administering a Linux, OS X or Windows (with Cygwin) machine remotely via a web browser. Based in Perl, it includes modules for many of the popular tools on the platforms: Apache, IPTables, Samba, MySQL, PostGreSQL. It’s a very handy way to familiarize yourself with the administrative options available with those tools, and often offers a simpler and easy to use interface than the tool itself does.

Powered by WordPress. Designed by Woo Themes

This work by Ted Roche is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 United States.