Project information
http://drupal.org/project/views_datasource
Current status:
Doing summer exams
Complete Atom support
Completed FOAF and hCard renderers and parsers, rolling 5.x and 6.x 2nd alpha release.
1st alpha release made; working on 2nd alpha release including FOAF and vCard renderers.
@TODOs:
Modify the devel generate-users script to insert values for profile fields as well.Go through the JSON and XML specs and see what characters need escaping as node content can contain pretty much any character.Delve into Views 2 options forms especially the one for the table view type to see how to write a form to allow mapping of view fields onto JSON or XML elements.(not needed right now.)Delve into the latest Views 2 beta - Views 2 beta 3 and see what cool stuff merlin has added to this release especially more default view types which means more code examples.- Grok the Drupal unit test framework and start writing unit tests for my plugins.
- Grok Views 2 relationships
- Figure out if it's possible in Views 2 if it's possible to dynamically add an icon in a normal view that links to a different view style - the way the RSS plugin works.
- Have a think about the stuff I should expose in the module beyond the main functions...access control is a no-brainer, but should there be a hook_node_xml or hook_node_json just like RSS does so modules can prepare their nodes to be serialized into these formats.
Creating test data
I modified devel module's devel_generate.inc script so it also generates data for user profile fields I've defined. Here is the modified devel_create_users function:
<?php
function devel_create_users($num, $kill) {
$url = parse_url($GLOBALS['base_url']);
if ($kill) {
db_query('DELETE FROM {users} WHERE uid > 1');
drupal_set_message(t('Users deleted.'));
}
for ($i = 0; $i < $num; $i++) {
$uid = db_next_id('{users}_uid');
$length = rand(4, 12);
$name = devel_generate_word($length);
$pass = md5(user_password());
$mail = $name .'@'. $url['host'];
$now = time();
db_query("INSERT INTO {users} (uid, name, pass, mail, status, created, access) VALUES (%d, '%s', '%s', '%s', %d, %d, %d)", $uid, $name, $pass, $mail, 1, $now, $now);
/*FOAF Profile */
$full_name_array = str_split($name, rand(1, $length - 2));
//fid 1:
$full_name = ucfirst($full_name_array[0]).' '.ucfirst($full_name_array[1]);
$titles = array('Mr.', 'Ms.', 'Mrs.');
//fid 3:
$title = $titles[rand(0, 2)];
//fid 4:
$nick = $full_name_array[0].substr($full_name_array[1], 0, 1);
//fid 5:
$mbox = $mail;
//fid 6:
$mbox_sha1sum = sha1($mbox);
//fid 7:
$openid = $nick.'.openid.drupal.org';
//fid 8:
$homepage = 'http://www.'.$nick.'.org';
//fid 9:
$weblog = 'http://'.$nick.'.blogspot.com';
//fid 10:
$img = 'http://www.'.$nick.'.org/me.jpg';
//fid 11
$phone = strval(rand(18005554444, 19000000000));
//fid 12
$member = 'Drupal';
//fid 13:
$jabberID = $nick.'@gmail.com';
//fid 14:
$msnChatID = $nick.'@hotmail.com';
//fid 15:
$workplaceHomepage = 'http://www.drupal.org/project/views_datasource';
$profile_field[1] = $full_name;
$profile_field[3] = $title;
$profile_field[4] = $nick;
$profile_field[5] = $mbox;
$profile_field[6] = $mbox_sha1sum;
$profile_field[7] = $openid;
$profile_field[8] = $homepage;
$profile_field[9] = $weblog;
$profile_field[10] = $img;
$profile_field[11] = $phone;
$profile_field[12] = $member;
$profile_field[13] = $jabberID;
$profile_field[14] = $msnChatID;
$profile_field[15] = $workplaceHomepage;
for ($j = 1; $j <= 15; $j++) {
if ($j == 2) continue;
db_query("INSERT INTO {profile_values} (uid, fid, value) VALUES (%d, %d, '%s')",
$uid, $j, $profile_field[$j]);
}
}
drupal_set_message(t('!num_users created.', array('!num_users' => format_plural($num, '1 user', '@count users'))));
}
?>FOAF use cases
I've been perusing d.o, Social Networking Sites on g.d.o and modules like buddy list and user relationships to get a feel for some use cases for FOAF export from a view. Here are 3 I've come up with:
- (simplest) A view displaying a list of user profiles or user content profiles created by nodeprofile or content_profile or usernode or any module mapping content to users, as long as FOAF properties like name mbox, memberof, weblog, workplaceHomepage are defined. A FOAF document is generated containing each user as a foaf:Person with these properties.
- Same as above but in this case if the buddy list user relationships module is installed, get the related users for each user and add them and their FOAF properties in a foaf:knows element.
- A typical list of nodes with each node authored by a particular user - pull out the users' details into a FOAF document and link the document (node) created by the user with a foaf:made. element.
Any comments on these three or suggestions for other cases?
Description
Views Datasource is a set of plugins for Drupal Views for rendering node content in a number of shareable, reusable formats based on XML, JSON and XHTML. These formats allow content in a Drupal site to be easily used as data sources for Semantic Web clients and web mash-ups. Views Datasource plugins output content from node lists created in the Drupal Views interface in a variety of formats - XML data documents using schemas like OPML and Atom, RDF data documents using a vocabulary like FOAF, JSON data documents in a format like Exhibit JSON, and XHTML data documents using a microformat like hCard.
Views Datasource can be thought of as a userland complement to the work going on in the Services and RDF APIs as a site operator only needs to know Views to be able to create datasources in the different formats.
The project consists of 4 Views style plugins (and related row plugins):
- views_xml - Output as raw XML, OPML, and Atom;
- views_json - Output as Simile/Exhibit JSON, Canonical JSON, JSONP/JSON in script;
- views_rdf- Output as FOAF, SIOC, and DOAP;
- views_xhtml - Output as hCard, hCalendar, and Geo.
Plugins will be provided for both Views 2 (Drupal 6.x) and Views 1.x (Drupal 5.x)
Project schedule
May 26th - June 8th
views_xml - OPML, and Raw XML
views_json - Exhibit JSON and plain-vanilla JSON
views_rdf - FOAF rev. 1
views_xhtml - hCard
July 7th - July 21st
views_xml - structured XML using Atom schema rev 1
views_json - JSONP/JSON in script (http://code.google.com/apis/gdata/json.html)
views_rdf - DOAP, SIOC rev 1
views_xhtml - hCalendar
July 14th - July 20th
views_xml - structured XML using Atom schema rev 2
views_rdf - FOAF rev 2
views_xhtml - Geo
July 21st - August 3rd
views_rdf - SIOC rev 2
views_xhtml hCard rev 2
August 4th - 18th
views_xhtml hCalendar rev 2
Complete docs and tests.
Status updates
Status update for 2008-07-17
This week I have exams for my summer courses so I haven't spent a lot of time of time on the project except for working on Atom support. As soon as my exams are done I'll resume work.
Status update for 2008-07-10
Last week and this I completed the 2nd alpha release of the module and wrote a demo page with an Ext JS table widget consuming a Views XML datasource. This week I have to complete Atom support rev 1. Not currently blocked on anything.
2008-06-25
Just a quick update, will add more later today. Last week and this I finished the FOAF and hCard parsers and renderers , fixed Exhibit JSON output so it now validates without error, and added simple JSON output. Now I just have to roll the bits and docs into a 5.x release today. Next will be the 6.x release and then on to hCalendar and DOAP and the other formats. The completed code for views_rdf and views_xhtml is in the CVS repository here and here.
2008-06-10
I wasn't very productive this week and I missed the first milestone of getting working FOAF and vCard renderers out the door for the second alpha release on June 8th. But I'm going to pickup the pace and aim for Thursday (June 12th ) as the new release date. All the objectives from last week are still in front of me. The FOAF and vCard renderers themselves aren't hard to write; it's dealing with where the data will come from that's more complicated. Some examples:
- You create user profile fields with common FOAF properties like mbox nick, weblog and so on. You create a view to display this information. The FOAF output can then render this as a FOAF RDF/XML document with all your users. So far so good. This is the simplest use case (I think I should focus on this for the initial release but there's much more you can do with FOAF:
- You have a view showing a list of nodes created my multiple authors - users on your site. You might want to have a view with FOAF profiles for each author and linked to their node by the foaf:maker or foaf:publisher property. In this case I would have to get the node author, lookup their profile info and render that. More complex than the first use case but certainly very useful and in the spirit of FOAF which is about linking documents to human authors.
But I want to keep deadline slippage to a minimum so I will concentrate on use case 1 for Thus 12th and if there's time to do the 2nd case I'll try to complete it.
2008-06-04
I'm going to start posting status updates as comments in addition to adding it to the wiki page to make it easier for the admins to pick them up.
Last week and this I rolled the D5 Views 1.x alpha release of the views_json and views_xml modules, researched the FOAF spec, researched RDF serializers, and begun to write unit tests for my code. What I need and am working on a script for is a database of users with an array of FOAF properties - proper name, homepage, current project, IM nicknames - the FOAF spec, defines a large list of properties a person may have and in a real-world scenario a user would have a Drupal profile with at least some of these properties defined.
So what I'm doing is use devel to generate users, then create a user profile with the fields I want and then use a PHP script to insert data into the profile table for each generated user. So for the generated user crumdum I would split the username at some random place and come up with the proper name Crum Dum. I would then insert this into the profile table for the profile field Name. I'd repeat the step for other profile fields like IM nick, just basing the field value off some variation of the generated username. Maybe this could be cleaned up and rolled into a patch for the devel module user generation script.
This week I have to:
- Complete the unit test suite for my existing code. It's important to get unit tests in right at the first release so when you add new stuff and stuff breaks, you have your unit tests as a checkpoint so you know what broke and where.
- Add the plain-vanilla JSON (canonical I called it) rendering. I had a chat with some Drupallers who said they were looking for JSON output for a widget they were using.
- Add the FOAF renderer to views_rdf. I looked at the Arc and RAP RDF toolkits for PHP but I think they are way overkill for my needs. These toolkits are deisigned to create full-blown RDF end-points supporting parsing querying and serving RDF, and come with their own RDF datatypes and such. I just need to serialize a simple array to XML using the RDF schema, so I think I will just use SimpleXML and create the document by hand, using the FOAF project code as a guide.
- Grok and document Views 2 relationships. I have a feeling that Views relationships may be a solution to a problem I've been speculating about. Basically FOAF and RDF in general is all about relationships - linked data. It's ok to have a FOAF document of a Drupal user, but if that person say is a member of an organization or group or has some nodes referenced in their profile, then they would want these nodes included in their FOAF profile. Relationships might be able to help - create the relationships between the nodes in the Views UI.
2008-05-29
Last week and this I completed the Views2 D6 port of the proof-of-concept JSON module
Image may be NSFW.
Clik here to view.
and made an alpha release: http://drupal.org/node/262445
Image may be NSFW.
Clik here to view.
I put all the Views2 API stuff I learned in the Views2 doc wiki page:http://groups.drupal.org/node/10129 as well as some user documentation I wrote. I've been reading discussions where people have been saying that the Drupal API is moving too fast and that every major release means a major rewrite of their module and a continuous knowledge gap. Well, I see the point but personally I can agree with the other side which says that API changes are to make things better and easier. Although I had to research a lot of the new Views API which meant scrabbling around in source code files, the new API is a lot easier to write plugins for and you have good examples to follow because all of the default objects are plugins.
2008-05-20
I'm currently going through Views 2 learning the new features and documenting as I go. I'm also working on a patch to update the doc stubs in Views 2 advanced help files. So far I've concentrated only on the user documentation but I want to finish that and start digging into the Views 2 API this week.