Because everything’s better with bacon

comprar sildenafil viagra prijs pilule levitra tadalafil moins cher vendo viagra vendo cialis cialis te koop acheter cialis sur internet acquisto viagra senza ricetta medicament cialis levitra italia vardenafil generico prix de cialis compro levitra viagra sans prescription medicament levitra acquisto levitra vardenafil bestellen trouver du levitra cialis pharmacie levitra sur internet generique du viagra sildenafil bestellen compro viagra viagra kosten comprar cialis cialis venta libre commander kamagra compra viagra acheter cialis en belgique ordina levitra kamagra te koop viagra pharmacie pharmacie en ligne acquisto viagra on line levitra france impuissance erection cialis kauf achat cialis 20mg levitra sur le net viagra donne generische levitra comprar cialis generico viagra ricetta acheter tadalafil commander du cialis acheter cialis internet viagra farmacia costo levitra cialis ohne rezept cialis vente libre viagra quanto costa levitra en pharmacie posologia viagra acquisto viagra zithromax generique ordina viagra acheter isotretinoine viagra rezeptfrei tadalafil generique comprar vardenafil generique du cialis commander du viagra vente levitra acheter kamagra 100mg generische viagra achat cialis propecia prix viagra dosaggio tadalafil 10 mg levitra generico compro levitra achat viagra en ligne acheter kamagra kamagra pharmacie aquisto viagra acheter cialis en espagne trouble erection viagra ordonnance cialis donne vente viagra cialis receta cialis vente en ligne vendita levitra viagra recensioni acheter zyban kamagra oral jelly acheter cialis en pharmacie acheter finasteride viagra te koop levitra venta cialis ricetta medica vardenafil generique sildenafil receta acheter cialis pas cher pastilla viagra viagra ricetta medica medicament impuissance comprar levitra generica impotenza sessuale tadalafil precio achat cialis generique viagra svizzera cialis belgique acheter clomid en france viagra cialis differenze cialis livraison rapide levitra rezeptfrei dysfonction erectile acheter du cialis cialis generico vente de cialis acheter cialis sur la net cialis effet secondaire kamagra rezeptfrei levitra precio acquisto viagra svizzera impuissance homme compro sildenafil prozac sans ordonnance pastilla sildenafil comprar viagra em portugal compro cialis levitra pharmacie prezzi levitra kamagra generique acquista levitra vendo viagra milano sildenafil rezeptfrei viagra fur frauen viagra effet secondaire cialis prescrizione

Tuesday, January 12, 2010

Array Sorting

I have this little sorting problem that had been bothering me for a month, but as it’s only a minor annoyance, I hadn’t spared the cycles to work on it. Last Friday I decided I needed an instant gratification project, and set about solving it.

I have a table that contains hostnames, cards, and the slots on the host chassis those cards are installed in. All datatypes are varchars. The sorting problem arises because I have some cards that are actually installed on other cards; those are referenced by a slot/subslot designation.

Here’s some sample data:

testytest=# SELECT * FROM cards ORDER BY card_slot;
hostname | card_slot |   card_model
----------+-----------+----------------
zucchini | 0         | card
zucchini | 0/0       | daughter card
zucchini | 0/1       | daughter card
zucchini | 1         | card
zucchini | 1/0       | daughter card
zucchini | 1/1       | daughter card
zucchini | 1/15      | daughter card
zucchini | 1/2       | daughter card
zucchini | 17        | something else
zucchini | 18        | something else
zucchini | 2         | another card
zucchini | 3         | another card
zucchini | 4         | another card
(13 rows)

I’d like to see 1/15 come after 1/2, and 17 & 18 come after 2, 3, & 4. (Like I said, it’s a minor annoyance. But still an annoyance.) I need to be able to sort both pieces in numerical order. I went through some weird gymnastics writing functions to split out & return each piece, but each of my solutions introduced other problems. (These instant gratification projects so rarely are, eh.)

Before I wrapped things up for the day, I read the PostgreSQL docs about array functions & operators. That percolated around in my brain and a solution came to me while I was out hiking over the weekend:

First, I reCAST string_to_array & made it return a set of integers:
CREATE OR REPLACE function foo(varchar(15)) RETURNS integer[]
AS $$
SELECT CAST(string_to_array($1, '/') AS integer[])
$$
LANGUAGE SQL;

Since Pg includes < and > array functions, I expected to be able to sort by the array my function returned:

testytest=# SELECT hostname, card_slot, card_model, foo(card_slot) AS sort_value
FROM cards
ORDER BY sort_value;
hostname | card_slot |   card_model   | sort_value
----------+-----------+----------------+------------
zucchini | 0         | card           | {0}
zucchini | 0/0       | daughter card  | {0,0}
zucchini | 0/1       | daughter card  | {0,1}
zucchini | 1         | card           | {1}
zucchini | 1/0       | daughter card  | {1,0}
zucchini | 1/1       | daughter card  | {1,1}
zucchini | 1/2       | daughter card  | {1,2}
zucchini | 1/15      | daughter card  | {1,15}
zucchini | 2         | another card   | {2}
zucchini | 3         | another card   | {3}
zucchini | 4         | another card   | {4}
zucchini | 17        | something else | {17}
zucchini | 18        | something else | {18}
(13 rows)

Voila. Works as expected on my small (~2000 rows) data set. It also fails as expected when passed bad data (eg something with a text string).

posted by gabrielle at 12:03 am  

Friday, November 6, 2009

Refactoring!

Last night at the hackathon,  we refactored one of our queries from my review of Refactoring SQL Applications.*

First, we had a duplicate field name in the original select.  Not a problem if you’re just doing a select, but if you want to create a table (temp or otherwise) from the data, it won’t work.  So we replaced the first num_rows with rows_in_bytes.

Also, reading over this 5 months after the original attemp, I realize it’s a lot clearer if we don’t use table aliases in the outer SELECTs.

Then, we got some advice from Greg Smith that we shouldn’t do joins on pg_class.relname – this can screw you up if you have different schemas with identical table names.  You want to use oids (which I’d always thought was not desirable, but I’m assured it’s ok if you’re doing it with the system tables – you don’t want your application to depend on them, though. :) )  So, instead, we match pg_namespace.oid with pg_class.relnamespace.

Selena’s illustration of how this works:
SELECT relname, relkind FROM pg_class
JOIN pg_namespace ON pg_namespace.oid = pg_class.relnamespace WHERE relkind = 'r' AND pg_namespace.nspname = 'public';

The new & improved version of the query can be found on the Pg wiki.

I wanted to compare the new query against the old, so I created a couple of temp tables containing the results… and discovered we had a couple of data discrepancies:  a few of our tables were listed twice in the original query results, with different values for num_rows, only one of which was correct for the current schema:

portal=# SELECT tablename, rows_in_bytes, num_rows FROM index_experiment_1
WHERE tablename IN  ('detectorid_count','stations','test_agg')
ORDER BY 1;
tablename     | rows_in_bytes | num_rows
------------------+---------------+----------
detectorid_count | 0 bytes       |        0
detectorid_count | 631 bytes     |      631
stations         | 22 bytes      |       22
stations         | 350 bytes     |      350
test_agg         | 0 bytes       |        0
test_agg         | 1386 bytes    |     1386
(6 rows)

portal=# SELECT count(*) from detectorid_count;
count
——-
0
(1 row)

portal=# SELECT count(*) from stations;
count
——-
350
(1 row)

portal=# SELECT count(*) from test_agg ;
count
——-
0
(1 row)

It turns out we’d run into the exact problem that Greg had warned us about.  The additional rows were from identically-named tables in other namespaces.

Find your namespaces:
portal=# SELECT nspname from pg_namespace order by 1;
nspname
--------------------
information_schema
pg_catalog
pg_temp_1
pg_temp_2
pg_toast
pg_toast_temp_1
pg_toast_temp_2
public
selena
wendell
(10 rows)

Find your data:
portal=# SELECT count(*) from selena.detectorid_count ;
count
-------
631
(1 row)

portal=# SELECT count(*) from wendell.stations ;
count
——-
22
(1 row)

portal=# SELECT count(*) from selena.test_agg ;
count
——-
1386
(1 row)

Note that these match the additional data from our original query.

Thanks, Greg!


* No, I haven’t finished reading it yet…I don’t read during the summer, I ride my bike.

posted by gabrielle at 6:35 pm  

Monday, October 19, 2009

PgWest: Sunday

We arrived at the conference site to find that the XML Data Warehousing had been canceled, so I spent that session in the Hackers’ Lounge attempting to continue work on pg_proctab, while getting kicked off the commie college wireless.

In Lists and Recursions and Trees, Oh My!, David Fetter gave us some example of old kludges to get row numbers out of Pg – “Not only is it slow, but it’s wrong” – but you may not notice that subtle wrongness in huge data sets.  This really illustrated the value of testing your data.

After lunch, I went to Josh Berkus’s 5 steps to PostgreSQL Performance Tuning.

He gave us some rules of thumb for figuring out how much RAM & CPU you need, but also recommends hiring a hardware geek to design your system for you – because vendors lie. :)   Try hardware out before you purchase it, or definitely test them within the warranty period.  And, here’s another use case for pg_proctab (other than my own amusement):  capacity planning.

Tip:  Don’t use autovacuum for data warehousing applications, or where you have large number of writes happening at once.  Manually vacuum those.

(An additional tip from me:  if you’re using linux, try increasing the default readahead buffer from 1024K to at least 1M for an ~80% performance improvement.  See our [in]famous file systems talk for the graphs to back this up.)

Thanks for another wonderful conference experience, PgPeeps!  See you again soon!

posted by gabrielle at 3:46 pm  

Monday, October 19, 2009

PGWest: Saturday

This past weekend was the 3rd annual PgWest.  The conference moved up to Seattle this year, and I think it was the biggest it’s ever been.  As usual, there were more interesting talks scheduled than I had time to attend.  (This is the 21st century;  where’s my time machine?)

For my first tech conferences a few years ago, I only went to sessions that were meaningful for my job.  I’ve since had a much better time (and learned more) by choosing which sessions I’ll attend based on the following criteria, in this order:
1) topic interestingness
2) speaker interestingess
3) relevance to my job duties

(See Tips #1 and #2 in Skud’s recent Ten tips for tech conference attendees post.)

So, right out of the gate at PgWest, I’m in a python talk* – Adrian K’s (of LinuxFestNW fame) discussion on Dabo.  Dabo’s a python desktop framework;  I program primarily in Perl, and I’ve never touched a desktop app.  Adrian’s example project was a management system for a plant nursery, which I *do* understand, so I had a point of reference into the material (the methods & options used to track plants made sense to me).  I really wanted to talk to him more about this app, but never caught up with him.  (The hallway track felt kind of rushed for me this time.)  I got a good idea for form validation – if user tries to enter a blank value where one is not allowed, they get a pop-up immediately and the original text (if there was any) is put back in the field, forcing the user to accept the original input or enter something new before they can proceed to the next field.  This is a step up from giving the user the error message after they’ve submitted the form.

Next we were on to JD’s keynote, featuring the usual heckling of and by the podium.

Then Mark’s & my talk about pg_proctab, which ended with some live demos & some audience participation, the way I like it.

A bunch of us went to lunch at Honeyhole Sandwiches, where I tried the “Texas Tease” – BBQ chicken.  The sandwich was excellent.  I *highly* recommend the fries.

Scott Bailey’s Temporal Data talk was *packed*.  He talked about the “period” datatype, featured in both his own (Chronos) and Jeff Davis’s PgTemporal project.  You can do unions & intersects on time periods.  I am thinking this would be a useful datatype for searching large tables of log entries.

Based on Scott’s talk, I decided to go to Jeff’s “Not Just UNIQUE” talk, because he would be discussing this in a little more detail.  This meant I missed the session on backup & recovery.  (See comment above about more material than I can fit in my schedule.)

I spent the last session partly in the hackers’ lounge, working on some pg_proctab wrapper scripts with Mark.

Then it was off to the EDB-sponsored after-party, where I caught up with Lloyd Albin, who spoke at PDXPUG about a year ago.  He brought me up-to-date on the work he’s done on the project, including a twitter feed to let clients know of updates, which I think is really cool.

*Which I was late to, because we were installing the snacks in the Hackers’ Lounge (thanks, Mark!)

posted by gabrielle at 3:35 pm  

Thursday, October 15, 2009

PDX.pm meeting notes

Jeff (aka @duckyd) gave a presentation about CPAN Awesomeness at last night’s pdx.pm.

Slides are on github!  Your homework is to find them. :)   Here are some highlights.

There are over 16K modules on the CPAN as of 11 Oct 2009.  Wow.

Jeff’s recommended changes from the default cpan shell configuration:
- auto_commit 1
- prerequisites_policy follow
- build_requires_install_policy yes
- prefer_installer MB (Module::Build)
- change your make_install_make_command and mbuild_install_mbuild_command to include your sudo command.

Spend 10 minutes & give something back to CPAN every time you install a module:  Simply set up CPAN::Reporter!

Recommendations:
- make sure that you set cc_author to ‘no’.  (In the latest version, that’s the default.)
- set it to prompt you to edit/send the report if the tests fail.  This way you can judge if the failure is due to your own
boneheadedness.
- you can set the transport value to a file to run reports without sending them.
- for help:  perldoc CPAN::Reporter::Config

For automatic continuous testing, set up CPAN::Reporter::Smoker.  (Doesn’t actually install anything, justs runs tests.)

Recommendations:
- don’t run it as root;  you are the canary in the coal mine.
- create a dedicated user that has essentially no privs on your machine.
- run it on a separate Perl install (core modules only).
- this is a cool place to use that RAND option for prefer_installer in CPAN.

Another cool tip that I REALLY DIG because I have systems with multiple perls & users associated with them:
Set your shebang line to:

#!/usr/bin/env perl

posted by gabrielle at 6:17 pm  

Tuesday, October 13, 2009

My picks for PgWest

(I’ll be missing Friday’s tutorials.)

Saturday:
9am:  Jeff Davis:  PostgreSQL, Extensible to the Nth Degree.  Jeff’s talks usually melt my brain, and I like that.
10:15:  Conference Keynote.
11:30am:  Mark Wong: pg_proctab.  Turns out I’m giving this talk with Mark, even though my name’s not on the schedule.  I should probably show up.
1:45pm:  Scott Bailey:  Temporal Data or Magnus Hagander:  Secure PostgreSQL Deployment.  There will be a coin toss.
3:00pm:  Kevin Kempter:  Backup and Recovery.  There’s always something else to learn about this topic.
4:00pm:  Bill Karwin:  Practical Full-text Search.

Sunday:
9:00am:  Aaron Sheldon:  XML Data Warehousing.
10:15am:  David Fetter:  Lists and Recursion and Trees (Oh, My!)  I want to learn about Windowing functions, new with 8.4
11:15am:  Matt Smiley:  Basic Query Tuning Primer.  Another topic I could stand to learn more about.
1:30pm:  Tossup between David Wheeler:  pgTAP Unit Testing Best Practices and Josh Berkus:  5 Steps to PostgreSQL Performance.  I’ll probably go to Berkus’s talk because Wheeler is a sport about repeating his talks for PDXPUG.

Other fun stuff:

The Hacker lounge will be open for two days of geekery:  7:30 am – 4:30pm Saturday, and 9-4 on Sunday.
EnterpriseDB has stepped up to provide entertainment after the Saturday sessions.
I haven’t heard if there are Lightning Talks, but I have a couple of ideas for one.  You should too.

See you there!

posted by gabrielle at 5:24 pm  

Thursday, October 8, 2009

Are you going to PgWest?

At a loss for what to do next weekend?  Grab your rain gear & head on up to Seattle for PgWest 2009.

There’ll be three days of talks & tutorials plus a hackers’ lounge.   After-party plans are nebulous at this time, but we are researching options.  (Psst–pub crawl!)

Come join the fun!

At a loss for what to do next weekend?  Grab your rain gear & head on up to Seattle for PgWest 2009: http://www.postgresqlconference.org/2009/west/.

Three days of talks & tutorials http://www.postgresqlconference.org/2009/west/schedule plus a hackers’ lounge.  http://wiki.postgresql.org/wiki/Hackers%27_Lounge.   After-party plans are nebulous at this time.  (Psst–pub crawl!)

Come join the fun!

posted by gabrielle at 5:07 pm  

Wednesday, September 23, 2009

PDXPUG Patch Review Party

(just in case you haven’t read about it yet):

http://pugs.postgresql.org/node/584

posted by gabrielle at 11:30 am  

Thursday, July 2, 2009

pdx.pm code sprint #1

Hacking on 5.10.1 was the plan…that happened for a couple of us. :)

Duke proposed a PDX.pm sprint to work on 5.10.1:
“I think if everyone learns how to

a) get a copy of the perl git repo
b) keep it in sync
c) run the perl test suite, including running a single test at a time
d) submit a small documentation patch

then we will have a great start.”

This is perfect for a first sprint: “small, solve-able tasks” that will get everyone up & running, plus have the potential to actually be productive.

Your first code sprint with a new group of people is like the first day on the job…except nobody realizes that we’re each FNGs*. It can take some time to figure out how to work together.

It’s really great if people get the stuff that’s going to take time from the code sprint out of the way beforehand. For example, cloning the perl5 git repo (step a):

:::=>git clone git://perl5.git.perl.org/perl.git

(Took me 13 minutes.)

Duke suggested some advance reading as well – how to use the repo (cd perl; perldoc pod/perlrepository.pod) (you will need to install perl-doc if you don’t have it; it was not installed on ubuntu.)
(more…)

posted by gabrielle at 10:38 pm  

Saturday, June 20, 2009

OSBridge Recap

This week I attended Open Source Bridge here in Portland.

Typically, I managed to miss the keynotes both days. There is something about conferences which makes me sleep through my alarm.

It was really hard choosing which talks to attend. The results of the coin toss:

Wednesday:
Tcl/Tk: Grandpa might be old, but he can still kick your ass! I went to this primarily because I use Expect so much. (Well, I use Expect.pm, but I remember my roots.) Webb gave a good intro to Tcl/Tk (”Tickle-Tea-Kay!”) despite some initial technical difficulties. I finally figured out the brackets vs braces variable expansion.

Then I gave my talk. Thankfully, Impress did not surprise me. I now have my unicorn badge.

Spindle, Mutilate, & Metaprogram: This was really cool, although it seemed similar to things that came out of the Perl community a few years back. I’d like to see a throwdown between Markus Roberts & Damian Conway.

Assholes are killing your project: I only managed about 20 minutes of this talk before I got too depressed & had to leave. Sorry, Donnie! We’ll talk about this later.

I spent some time in the hall track & then hit the yoga session. This was an excellent pick-me-up after a day of talking and brain-filling, and set me up for my BoF and then some time at the pub.

Thursday:
Arrived too late for chromatic’s Intro to Parrot so hung out in the speaker lounge and watched Andy and Irving’s run-through of Virtualize vs Containerize: Fight! I love the mashups.

Next up was Emma McGratten’s Ask Forgiveness not Permission, which had a lot of excellent reasons (financial & otherwise) for using open source, but not many tips on how to subversively bring it into your organization. I’m sure I know someone who could give a talk about that. :cough:

Lunch today was the excellent KOiFusionPDX Food Cart! They came to the conference site & provided excellent korean tacos. (Yeah, I know, sounds weird – but TRUST ME.)

Speaking of trust…Trust the Vote sounds like an excellent project. Unfortunately the question period started devolving into political discussion, and I didn’t want to just dive right in there and ask them why the hell they’re using MySQL instead of PostgreSQL.

Maria Webster got her unicorn badge for Faking it Till I Make It. Check out her blog to see what geeky women are up to.

bzr vs git smackdown with Selena & Emma. I’ve already made up my mind (git all the way!), but it’s good to listen to alternatives.

The Meditiation for Geeks session didn’t go too well for me, because I was so tired that any time I got close to The Zone, I almost fell over onto a fellow PostgreSQL Smurf. Still, the yoga & meditation sessions are a great way to unwind prior to the post-con socializing & I’d like to see more of this.

Pg took over the room & had our PostgreSQL BoF, which replaced the regular PDXPUG meeting.

Josh Berkus was riding a bicycle around town, which made me inordinately happy. I want to see if we can provide more bikes for attendees next year.

Friday: The Unconference rocked my socks:
1. Emma Jane’s “Playing with yourself” about Open Source documentation teams. I am even sadder that I missed WOSCON. This got me totally excited to contribute to docs. (Especially for certain Perl modules – but that’s a discussion for another post.) Highlights: the conference team is working on a style guide, and a library of personas (which isn’t public yet)

2. I signed up with DayOn, a local volunteer effort. This will be fantastic once we can get people trained in what’s actually reasonable to ask for.

3. I did a Network Management Basics talk (”FCAPS: What the hell?!?”) with Ua and Adam. We talked about the FCAPS model & where various tools we use fit. A very high percentage of them are rrdtool-based, so we talked about that a bit as well. Adam showed us his munin install. I keep trying to find other people in town who are as into Net Management as I am…I sort of feel like I need a 12-step program sometimes. On the way out, Ua proposed a Super-Sekrit project which we’ll start working on in September. (Excitement!)


I saw up-close what it took to put on this conference and I’d like to congratulate the organizers on their success! Great job, and can’t wait until next year!

posted by gabrielle at 5:52 pm  
Next Page »