Making Government Transparent and Accountable
The Sunlight Foundation uses cutting-edge technology and ideas to make government transparent and accountable. Underlying all of our efforts is a fundamental belief that increased transparency will improve the public's confidence in government
The Sunlight Foundation Blog
-
Introducing Sunlight Live
As the Open Government Directive was announced in a live webcast back in December, Sunlight tried something a little different by covering the event live in a variety of formats at once.As is a norm around here, we basically just got a lot of people in a room, tried a bunch of stuff and paid attention to what seemed to work. At the end of the announcement we simultaneously had a tweet stream from across the open government community going, a live blog, and a Google Wave. We threw the obligatory word cloud at it, sent email blasts, and followed up with blog posts about the Directive’s many components.
It was fun and seemed to be pretty effective. And it also got us thinking…
What if we were able to “cover” live events in a new way using government data that we’re able to compile and connect it to political events and personas of the day?
Today we’re going to take this idea to the next step by beginning to connect government data such as campaign contributions or lobbyist meetings to a political event in real-time. As Republican and Democratic leaders come together to debate health care in a public forum, Sunlight is going to provide an alternative to the mainstream media’s coverage. In a replicable pilot we are calling Sunlight Live, our team will connect data such as the aforementioned lobbying contributions or “revolving door”connections the meeting’s participants may have, and put them right next to the video feed, as any particular politician is speaking.
http://sunlightfoundation.com/live
We think Sunlight can offer a unique live perspective on the debate in the midst of the media frenzy, by focusing not on the merits of health care reform, but on the money, connections, and influence data to which we have created access. In addition to displaying data from Sunlight and its grantees’ projects, our staff will once again be live blogging, facilitating online conversation via Twitter, and engaging the open government community in research as the debate unfolds. We don’t yet know exactly what we’ll need or what will work best …but that’s the point.
We’ll be getting things started at 10am with the beginning of the meeting. Hope you’ll join us!
-
OGD /Open Pages Come to Life
Meeting or surpassing the White House’s deadline, the 20 agencies we monitored launched their /open pages by this past Saturday in accordance with the Open Government Directive. (See Sunlight Lab’s /open page tracker; also ProPublica’s transparency tracker).
The White House also revealed its Open Government Dashboard, which monitors 29 agencies for compliance with the OGD. The timely creation of this Dashboard fulfills another promise contained in the OGD.
What’s notable about the White House’s Dashboard is that it helps hold agencies accountable by identifying the 4 agencies that have yet to fully comply with the OGD: the Nuclear Regulatory Commission, the Office of Personnel Management, the Council on Environmental Quality, and the Office of the US Trade Representative, in addition to indicating the 25 that have fully complied. These 4 agencies have yet to release all three high-value data sets. The Council on Environmental Quality receives an additional warning light for having failed to assign a senior official to ensure data quality.
We are in the preliminary stages of looking at what’s on these /open websites, with four questions initially springing to mind:
- Does each agency’s website contain everything set forth in the OGD?
- Do they exceed those expectations?
- Are the websites designed well?
- Are they making progress towards developing their Open Government Plan?
Both Intellitics and the General Services Administration have put together links to each agency’s pages (and RSS feeds) for gathering feedback and engaging in discussion with the public about the Plan. GSA has also added additional means of contact, specifically email and postal addresses.
In addition, GSA has there is an additional tool available: a wiki — the open government playbook — that aggregates a lot work done by the government and those outside the government on transparency. Its purpose is to “serve as a useful directory to [OGD] resources” — and invites everyone from government officials to members of the public to contribute. It is a great idea. (We had previously gathered all of our OGD resources on our separate wiki page.)
We’ll have a lot more to say in the upcoming days and weeks, particularly as we dig into the /open pages and the OGD Plan.
Before moving on, I must note that the White House took a risk in publicly setting a deadline for creating these /open pages. Meeting this deadline may not result in a lot of favorable media attention, but had agencies failed to do so, the White House may have been subject to a lot of criticism. The architects of the OGD deserve credit for taking a risk, for being willing to risk public failure in order to make something good happen. At first glace, they made it happen. Congratulations.
We’re going to take a hard look at these webpages to see how well they satisfy the details of the OGD, what improvements should be made, and evaluate the emergence of the Open Government Plan over the next 60 days. There’s a lot more to do, but Saturday marks an important milestone.
Updated: the wiki is linked to from GSA, but is not run by them. I’ve been lead to believe that Lucas Cioffi is one of the volunteers leading the wiki effort.
-
Building a National Campaign for Transparency
What an exciting (and long and intense) few weeks it’s been.
When we got back into the office here at Sunlight on January 4th, we knew 2010 was the year we needed to build a national campaign of people calling for an open, transparent government everywhere across the country. We’ve known for months in fact, but honestly, we had no idea how people would respond when we put the word out.
Now, only three weeks later, a few very big events have unfolded, and each new event has created new opportunities – as well as more need for our collective action than ever before. In other words, it’s abundantly clear that we’re on the right track.
It’s time to start putting the pieces together, and we’ve updated our initial sign up page with a statement of need for a national campaign and our next 4 steps to launch it in March.
(Continue reading…)
-
Open Gov’t Directive: large data set release today
This evening, the White House issued a press release (included below) touting the release of nearly 300 data sets from 24 departments and agencies — in time to meet the first major deadline of the Open Government Directive. Also due today was each agency’s designation of a high-level senior official to be responsible for the quality of data disseminated through public venues (like USAspending.gov).
While it will take some time to examine the data sets’ contents and assess whether these are indeed high-value data sets — something our reporting team is on top of — what is undeniable is that this kind of information release would be unthinkable thirteen months ago.
Today’s actions provide concrete evidence of a major change in the attitude of the executive branch in favor of transparency. Only time will tell if the data lives up to this promise.
In 15 days, we can expect agencies to release their /open webpages. (We have suggestions about that, too). This is getting interesting.
-
Time Running Short to Meet First OGD Deadlines
Executive branch departments and agencies are under tight deadlines to comply with President Obama’s high-profile Open Government Directive [PDF], with the first round of actions due Friday, January 22. Additional steps must be taken in 15, 45, and 60 days.
Both the administration and watchdog organizations like the Sunlight Foundation are monitoring each agency’s compliance with the OGD. This will make for easy comparison among the agencies to see who is ahead of the curve.
Here are three milestones that agencies should now be meeting:
First, by this Friday, each agency should have completed the following tasks:
- Identified and published online in an open format at least three high-value data sets and registered those data sets via data.gov.
- Designated a high-level senior official to be accountable for the quality and objectivity of, and internal controls over, the Federal spending information publicly disseminated through such public venues as USAspending.gov or other similar websites. The official shall participate in the agency’s Senior Management Council, or similar governance structure, for the agency-wide internal control assessment pursuant to the Federal Managers’ Financial Integrity Act.
- Planned for the tasks due in the upcoming weeks.
Second, each agency should have established an OGD team.
That team likely should include people with the following backgrounds: senior leadership, information technology, new media, FOIA, legal, communications, strategic planning, and legislative affairs. It should be able to act rapidly and decisively, with its members able to work on multiple issues simultaneously. Now is the time to assess the team’s progress and compliance with the OGD.
Third, each agency should have developed a skeleton model of their /Open website, which must be operational by February 6.
Sunlight has held several meetings to discuss the /Open page, generated a list of suggestions for what it should contain, identified principles by which data on those pages may be evaluated, and will shortly release a wireframe model of a /Open website.
Subagencies have responsibilities under the Open Government Directive as well. They must help identify high-value data sets, pro-actively use modern technology to disseminate useful information, publish information online in open formats, and work to resolve their backlog of FOIA requests. It is likely that OMB will issue further requirements as to how government information must be made available online in a timely fashion.
The OGD is a high priority for the administration, as evidenced by the President’s Feb. 24 memorandum and the symbolism of the January 22 due date. It is likely that this initiative will broaden over time and continue to be a focus of government policy for years to come. Agencies should plan for robust compliance with the Directive.
The Sunlight Foundation welcomes the opportunity to assist agencies with meeting and exceeding the requirements of this Directive.
-
Don’t Give Grades Till the End of the Class
Kenneth Vogel over at Politico wrote this morning about a report put together by a coalition of watchdog organizations praising the White House for its openness by giving it an “A” grade.
The watchdog report speaks to the positive efforts undertaken by the administration to make the executive branch more open and transparent, and the highlighting of those efforts is all well and good.
But giving openness grades to the White House right now is like giving a grade for an entire class after the first few weeks. A professor wouldn’t give a grade for the whole semester because a student turned in quality homework in September and laid out a plan for how hard she is going to work over the next few months until the final. Encouragement, guidance, oversight… sure. But not a grade.
-
White House Asks for Help with Data.Gov and OGD Dashboard
The White House is soliciting feedback on Data.Gov and its Open Government Directive Dashboard. Here is the nub of their request for your participation:
1. Open Government Dashboard: The Open Government Directive calls for the creation of an Open Government Dashboard to measure progress and impact. Deputy Chief Technology Officer, Beth Noveck is looking for your input, including as to the metrics by which we measure success. Click here to participate.
2. Future of Data.gov: The Open Government Directive instructs all federal agencies to make available high-value data that promote national priorities and improve the lives of everyday Americans through Data.gov. Yet the current version of Data.gov is just the beginning. Chief Information Officer Vivek Kundra asks for your help in shaping the future of this key open government platform. As part of the Data.gov Dialogue, you can download the draft plans, submit a new idea, or comment on someone else’s. We look forward to Evolving Data.gov with you.
-
Treasury Imposing “Terms of Use” to Access TARP Data
In an astonishing move, the Treasury Department is requiring users to agree to “terms of use” before they can access and download TARP Transaction Reports in spreadsheet format. This requirement undermines the rationale for releasing the data, may implicate federal law, and is simply foolish. It also sets a bad precedent.
The TARP Transaction Reports, which contain detailed information on the government’s Wall Street bailout, are available in PDF format going back to November 2008. Only now are they being published online in a much more useful spreadsheet format, known as XLSX, thanks to President Obama’s Open Government Directive. The Report accompanying the Directive lauds the release of these weekly transaction reports as “improving transparency of federal bank supervisory activities as well as the investment activities of financial institutions.”
The “terms of use” that accompany the TARP reports, reproduced in full at the bottom, require users to:
- Affirm that they have read and understood the site’s privacy policy and disclaimers,
- Acknowledge that the terms of use may be modified at any time, which the user agrees to accept,
- Clearly cite Financialstability.gov on all reuses of data accessed or retrieved from Financialstability.gov,
- Clearly state that neither Financialstability.gov nor the U.S. Government vouches for the data or analyses derived from the data after it has been retrieved from the website.
Users who refuse to click “accept” are not permitted to download the data. (Continue reading…)
-
A Watershed Moment in Transparency and Accountability
I was out of town earlier this week when the Open Government was released and so I am just now weighing in with a few thoughts. Without being too over the top (OK maybe I am) I think this Directive potentially represents a watershed moment for democracy, the likes of which can forever change the relationship between the government and the public it serves.
This Directive acknowledges that the Internet is the right, proper and primary medium for communicating information to the public, and that “the public” is key in helping determine the policies, directions and priorities for government’s work. In doing this, the Directive requires each executive agency to create an online portal designed to promote important agency data and provide a place for citizens and agency officials to work together.
Every citizen – from web developers to journalists to a real estate agent in Kentucky or mother in Colorado — has something to gain from this new initiative. If you care about the health of your children, the safety of your workplace, the crime in your streets, or corporate accountability you will have new information to inform yourself and discuss with those who represent you.
Business will have a lot of new data to base their decisions on. Real estate agents and developers will be able to see migration patterns by income when the Internal Revenue Service releases their database of tax-filer migration from county-to-county or state-to-state. This will aid in decisions on where to build, what kind of market to expect and what type of people live where. Similarly, data released by the Department of Housing and Urban Development on public housing will not only help reveal slum lords, but also help renters or buyers locate Section 8 density when deciding on a neighborhood to move into.
Here are some more examples:
• The Federal Aviation Administration (FAA) plans to release a real-time online database providing up to date information on flight delays and cancellations. This database will take existing FAA data and combine it with data from the National Oceanic and Atmospheric Administration (NOAA) to allow the public to be able to easily access online the status and causes of airport delays and cancellations.
• The Department of Homeland Security (DHS) is creating Virtual USA, an information-sharing system that will reduce lag time in responses to emergency situations. This system will likely save lives and empower local authorities with better information to reduce the costs of emergencies and disasters.
• The Department of Veteran’s Affairs (VA) will release raw data and “report cards” on Veteran’s Administration hospitals empowering current and former military service members and their families to know the quality of care that they are getting when they go to a given VA hospital.The Directive will also help citizens hold their government accountable through new avenues of government disclosure and citizen engagement. For example, the Department of Justice is collecting and publishing agency reports on their Freedom of Information Act (FOIA) compliance. Each agency report will include information on how long it takes to process FOIA requests and which agencies are most successful at fulfilling FOIA requests from the public (and consequently which agencies are not as well).
The Directive also requires each agency to allow citizens to request data to be released by agencies and placed online. This process will enable citizens to continue to press officials to make their data as open as possible, and that call for more information will have an affect far beyond the executive branch.
The General Services Administration (GSA) will be releasing a full database of all federal advisory committee members that can be mashed up with lobbying records and contribution databases to show the influence that resides on these important bodies – and we can similarly expect Congress, states and municipal governments nationwide to feel pressure to release information. That is, if the public demands it.
It will be our responsibility as citizens to monitor the data quality reported by agencies and the timeliness of the reporting requirements. Just like all previous open government moments, this one will require constant public engagement to ensure that will be all that it can be.
Sunlight is bullish on this Directive because we believe it will make permanent the idea that open government means an online government. In the digital age how can it mean anything else?Even if you don’t realize it yet, the plan laid out in the Directive will impact you, so we hope you’ll join us during this exciting to begin a this new dialogue with government. What we have now is a plan. And it is a plan that will require public engagement to ensure that the policies it lays out are enacted and undertaken to make our dream of an open government a reality.
-
Three Misconceptions about the Open Government Directive
There are three common misconceptions about the Open Government Directive that I’d like to clear up.
1. “It was late.” Not so, unless you’re talking about hopes rather than requirements. This misconception comes from the language in the January 21st memo, which directed the “Chief Technology Officer…to coordinate the development…within 120 days, of recommendations for an Open Government Directive.” The deadline of May 21, 2009, was for “recommendations,” not for the directive itself.
2. “The three new datasets must be never-before-released.” This may be the confusion most likely to cause controversy, since this criterion for newness will be used to judge how well agencies fulfill the directive. This hinges on how you read this sentence:
These must be data sets not previously available online or in a downloadable format.
You could read the sentence to mean that the data must be hitherto neither online nor downloadable, which would imply that all the new datasets released to fulfill the requirement must be brand new to the public. This seems unlikely, since no public dataset available in a downloadable format to the public would be “not previously available online.” In other words, the likeliest intended meaning of the sentence is agencies can publish datasets that have been published poorly before as data for the first time, and have that count.
In that case, DOJ publishing the FOIA performance spreadsheets would count as one of their three datasets required to be published within 45 days, since they’ve only been published as documents until now.
Parentheses may help elucidate the confusing sentence structure:
These must be data sets (not previously available online or in a downloadable format).
These must be data sets (not previously available online) or (in a downloadable format).
3. “High value datasets aren’t defined.” I spent part of the rush of Tuesday’s release thinking this was the case. From the attachment to the directive:
High-value information is information that can be used to increase agency accountability and responsiveness; improve public knowledge of the agency and its operations; further the core mission of the agency; create economic opportunity; or respond to need and demand as identified through public consultation.
