Doc Searls, author and Senior Editor of the blog Linux Journal, interviews Sunlight’s Greg Elin in an article about open source in politics and government. Here are the choice parts from Greg describing Sunlight’s work, the data that backs it up, and the future of it all:

Almost all of our projects and funded projects are open source — though sometimes our code is a bit hacked so it takes a while to release it. Nearly every group I know is completely invested in open source: MySQL, PostgreSQL, Apache… The frameworks are being rapidly adopted: Rails, Django, Symfony…

The work I’m most interested in these days is dynamic-scripting — what I think about as “flow-and-go” data sets instead of what Jeff Jonas coined as “rack-and-stack” data sets. Dynamic scripting is Unix pipes! That is, every application does input and output. We leave the world of databases-make-reports and enter the world of RSS-flows-in and RSS-flows-out.

Two examples of flow. A Sunlight database, LouisDB.com, scrapes the Congressional Daily Record daily, transforming it into XML. Garrett Schure (Sunlight Labs developer) and Josh Ruihley did a word count algorithm on the Congressional Record to come up with Congress’ “Word of the Day” and the microsite http://capitolwords.org — which goes back to 2001 and has an RSS feed, API, and a widget people can put on their site. Louisdb.com makes it easier to search the Congressional Record — and now there’s a script boiling it down into tweetable content that others can use, too. Second example, from MySociety: TheyWorkForYou. It provides profiles of what Members are doing in Parliament by parsing the Parliament’s daily record and votes. Lastly, many sites rely on the work of Josh Tauber’s http://govtrack.us b/c. Josh scrapes all sorts of data on bills in Congress and transforms it into XML. Josh’s data is open and so also is his code. It’s a tremendous contribution.
…
Programmers and technologists who grew up with the web and with open source have been entering the political and e-government arena the past several years bringing with them the tools and practices of open source and Web 2.0. They are collaborating with — and sometimes competing with — existing technologists who were often activists who learned spreadsheets and databases and desktop publishing and then the web to communicate their message. So we are seeing a geek-i-fication of everything from campaigns to good government groups to government itself. More open source. More frameworks. More collaborative communication among individual developers. It’s uneven, it’s bumpy, but it is definitely happening. The tipping point has occurred now in politics and government — the question remains only where the tree is going to land.

0 Comments

Greg Elin — Sunlight’s Chief Data Architect — did a fascinating interview with Jon Udell last week. I have the daily benefit of Greg’s insights and so I want to share this very insightful interview so you can too.

Jon has blogged about the interview here. Udell is an author, information architect, software developer, and groupware evangelist himself. He writes a monthly column for the O’Reilly Network. It’s worth a regular read.

Update:Canada’s DataLibre ran a good interview of Jon Udell on August 6 that is really worth the read.

0 Comments

We’ve been promising to introduce our Sunlight Labs more formally and today we’re doing that, along with the announcement of a really neat widget that we’re calling "Popup Politicians." Before you imagine the worst, like, Representative J. Dennis Hastert or Sen. John McCain or Representative John Boehner popping out of cake, take a look at what Greg Elin and Duncan Werner have developed — a web page plug-in that links the reader to information about who’s financing the lawmaker’s campaign, the lawmaker’s voting record, and their profile on Congresspedia. The widget appears as a small popup window when you mouse-over the little sun icon that appears at the end of the name.

We’re experimenting with the Popup on our site today. Check out all the technical information on the Sunlight Labs page.

What is it? Popup Politicians is an AJAX-powered widget which contacts a remote database here at Sunlight to retrieve links we’ve selected for a politician. The single Javascript that powers the mouse-over "bubbles" is served from Sunlight Labs server along with the data. When you load the page, the Javascript looks for Technorati-styled link tags for Members of Congress on the web page and then dynamically modifies found links to add the rollover action and a mouse-over bubble.

Sunlight Labs is readying various flavors of the widget for increased scalability. The basic widget can be added to a web site or blog by simply adding the Javascript and style sheet to the page’s headers and then manually adding a properly formed linked to each members of congress name where a popup is desired. Micah Sifry tried it last night on his personal blog and it worked. Other flavors include local server-side PHP code to automatically search and replace members of congress’s names with the necessary links. Sunlight has built a Drupal plugin that does this for our own site, www. sunlightfoundation.com, and also built a WordPress 2.0 plugin as well. Plugins for the major blogging and CMS platforms are planned and SunlightLabs is eager to find open source developers to help accomplish this and extend the the plugin. 

That’s beyond the extent of my knowledge. Check it out here. And email Greg Elin at gelin@sunlightfoundation.com for more information, particularly if I have left you thoroughly confused. And remember, we’re still experimenting with this, so all is not perfect yet.

0 Comments

Already the Sunlight Mash-Up Labs announced in May is striding toward my fantasy of one-click political influence disclosure. Last week, Lab Co-director, Greg Elin, guided me through the results of a week of "hacking" with Mike Krejci, lead programmer for The Institute of Money in State Politics. Supported by a small grant from the Sunlight Foundation, Greg went to Portland, Oregon and helped Mike begin work on The Institute’s "web services API".

The Institute tracks campaign finance data on some 18,000 state-office candidates each election cycle and now manages a database of some 14 million records spanning many years. Even though The Institute makes this data available via its respected FollowTheMoney.org web site — which is pretty amazing when you think about it — the fact is getting at that data can be cumbersome, especially when you are on a different web site. As it is now, looking up information on your state candidate means leaving whatever website you are on and going to FollowTheMoney.org and searching through various pages to look up the data you want. 

Web services API changes this picture dramatically. According to Greg, a web services API (short for Application Programmer’s Interface), "is a machine-friendly interface to a web site’s underlying complex database and application."  By adding a web services API to their web site, The Institute is making it significantly easier for programmers at other web sites to dynamically incorporate The Institute’s data into their own web-based applications. And that means in the future you and I won’t have to change web sites to see the data that matters. It will already be there.

To give us non-ubergeeks a sense of this future, Mike and Greg mashed-up a few web page "widgets" which remotely search The Institute’s data. You can try one here. You can search by state, year, office, won/lost, party and even candidate without ever leaving the web page or even reloading the web page. Your search is automatically sent to The Institute’s API in the background which delivers the results dynamically into the page at which you are currently looking.

The ability to easily integrate data from one web site into another really changes the big picture. There’s simply too much data for a single entity to manage. It simply takes too long to bounce from site to site to research subtle patterns of influence buying. But allowing summary data, or detailed data, to more easily move between data silos creates the means to browse — and compare — hundreds of million data points simultaneously. Pretty neat.

Greg tells me Mike still has work to do before the APIs are ready for public release, but that Mike made enough progress they are ready for limited trials with The Institute’s partners. I can’t wait to make further announcements.

0 Comments

I’ve long had the fantasy of one-click political influence disclosure. Imagine pressing one button and finding everything you need and want to know about a member of Congress, or a corporation, labor union or individual trying to influence her. Web 2.0 technologies - Web services, API’s, XML, AJAX, RSS - now make that possible.

To speed up making this happen, this week we decided to create a small, informal "Mash-Up Lab." We are going to treat this as a pilot project for six months to experiment on our own and to provide ad-hoc technical support to nurture other mash-up projects — some of which Sunlight has already  nurtured, to realize a one-click future. These will be projects that strategically and tactically bring together nonprofit organizations, exemplary developers, and web-applications.

Heading this project will be Greg Elin, a software developer specializing in databases and interactive technologies. Greg is the creator of FotoNotes^TM, a very cool image annotation program. For 15+ years he has helped organizations articulate requirements and prototype new technologies, everything from NYNEX (now Verizon) to dot.coms to non-profits. Since 1998, he has independently provided database and technology services for a variety of clients including New York University and the United States Naval Research Labs. Greg has extensive experience working with institutional and distributed data. Maybe most important, he can communicate clearly about all this. He maintains a thoughtful weblog at http://duhblog.com.

Operationally we envision a combination of activities: undertaking our own tractable projects and prototypes on a monthly/weekly basis; convening informal gatherings to nudge organizations that have already begun to have conversations about data mashing; and interacting virtually and face-to-face (and providing matchmaking with developers) to groups needing advice and technical support. We will also create and facilitate "Do-It-Yourself How-To’s" on mashable resources.

We’ve already heard from a number of organizations which we think would benefit from this kind of work we can offer. Let us know if you know of more.

0 Comments