Daniel Ansari’s blog Random software musings

February 12, 2010

NHibernate caching solution for web farms

Filed under: NHibernate,Open Source — Tags: , , , — admin @ 11:08 am

If your application uses NHibernate in a web farm, there is already a second-level cache provider that you ought to be familiar with: NHibernate.Caches.SysCache2, which supports expiration based on SQL table or command-based dependencies.  Whilst this is a decent solution, it does cause your whole cache region to be invalidated when data changes in the table.

It may be more ideal for you to invalidate the cache on a per-object basis, i.e., if one web server in the farm receives a request that causes a cached object to change, the portion of the application running on that server should be able to expire that object from the cache of all other servers within that farm.

A technique that I have used somewhat successfully is a shared disk cache invalidation mechanism called FileSysCache, similar to the “File Update Model” discussed in this MSDN library article.  Unlike the technique in that article, however, ours does not suffer from the issues arising from SQL Server trying to access the file system.  Our method creates a zero-length file on the shared disk, together with a CacheDependency to that file, for each cacheable object.  It deletes that file when the cache is invalidated for that object.  Thus, any other servers holding a cached copy of that object will have a CacheDependency on the same file, and their cached copies will expire.

This happens behind the scenes for you as an application developer.  NHibernate will call the relevant methods in the caching provider; all you need to know is that to expire the object across the web farm, you just need to evict it from the second level cache, using:


To configure FileSysCache, put the following into your web.config:

    <section name="filesyscache" type="NHibernate.Caches.FileSysCache.FileSysCacheSectionHandler, NHibernate.Caches.FileSysCache" />

and also:

  <filesyscache path="C:\Projects\NHibernate.Caches.FileSysCache.Tests\bin\Debug\Cache">
    <cache region="foo" expiration="500" priority="4" />

I meant to make this post a few years ago, so I hope the information here is still relevant, and I hope it’s better late than never.

The code, including unit tests, may be found here.

Adobe Flex Builder 3 problems in 64-bit Windows

Filed under: Adobe Flex,Open Source — Tags: , , , , — admin @ 12:54 am

Making the move to Windows 7 Professional 64 brought about a plethora of problems, some minor and some not so minor.

Fortunately most of the troubles were shared with other developers, and information is there to be found using our favourite search engines.

If you’re a Flex developer, like me, you might not have as much luck.  If you’ve downloaded the latest (stable) Flex SDK—3.4.1 at the time of this writing—it’s not possible to set up your “Installed Flex SDKs”, in order to set this SDK as the compiler default from the Preferences dialog.  That’s because of an issue on current x64 Windows where controls won’t get resized once the nesting hierarchy of windows exceeds a certain depth, which was referred to from the only useful information I could find about this specific issue.  It’s also not possible to set many preferences, as certain project properties show up as blank, and certain buttons are missing.  It’s absolutely abysmal that Adobe hasn’t remedied these issues; how does it expect developers to do any serious Flex development on 64 bit Windows that’s becoming mainstream?

Fortunately, there is a workaround, at least for the issues that affected me.  The solution lies in having a working copy of Flex Builder on a 32-bit Windows platform.  Flex stores these preferences in %USERPROFILE%\My Documents\Flex Builder 3\.metadata ( %USERPROFILE% is the environment variable with a typical value of “C:\Documents and Settings\{username}” on Windows XP, and “C:\Users\{username}” on Windows 7), and subfolders therein.

To configure your installed Flex SDKs, first unzip your Flex SDK to “C:\Program Files (x86)\Adobe\Flex Builder 3\sdks” (or wherever it is on your system); there is a file at the top level called flex-sdk-description.xml.  Open this up and make a note of the <name> element.  Next, open the file “%USERPROFILE%\My Documents\Flex Builder 3\.metadata\.plugins\org.eclipse.core.runtime\.settings\com.adobe.flexbuilder.project.prefs”.  This is where you’ll insert the setting for com.adobe.flexbuilder.project.flex_sdks, making sure you use the SDK name noted earlier.  My entire file looks like this (without the line breaks before “\r\n”):

#Wed Feb 10 17:49:32 EST 2010
com.adobe.flexbuilder.project.flex_sdks=<?xml version\="1.0" encoding\="UTF-8"?>\r\n<sdks>
\r\n<sdk location\="C\:/Program Files (x86)/Adobe/Flex Builder 3/sdks/3.0.0" name\="Flex 3"/>
\r\n<sdk defaultSDK\="true" location\="C\:/Program Files (x86)/Adobe/Flex Builder 3/sdks/3.4.1" name\="Flex 3.4"/>
\r\n<sdk location\="C\:/Program Files (x86)/Adobe/Flex Builder 3/sdks/2.0.1" name\="Flex 2.0.1 Hotfix 3"/>\r\n</sdks>\r\n

That fixes the first problem.

My other main problem was that I needed to debug the Flex application, but I was unable to add the launch configuration under Run/Debug Settings of the project properties—the New, Duplicate, Edit, and Delete buttons are missing and cannot be activated via the Windows shortcut keys.  If your project is called ProjectX, you’ll need to have a file called “%USERPROFILE%\My Documents\Flex Builder 3\.metadata\.plugins\org.eclipse.debug.core\.launches\ProjectX.launch”.  For me, the .launches folder wasn’t there, so I copied it from my 32-bit system.  You can create the .launches folder manually, though not from Windows Explorer (you may have to use the command line).  My file looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<launchConfiguration type="com.adobe.flexbuilder.debug.launchConfigurationType.flash">
<stringAttribute key="com.adobe.flexbuilder.debug.ATTR_APPLICATION" value="src/ProjectX.mxml"/>
<stringAttribute key="com.adobe.flexbuilder.debug.ATTR_DEBUG_URL" value="http://localhost/abcde"/>
<stringAttribute key="com.adobe.flexbuilder.debug.ATTR_PROFILE_URL" value="C:\Projects\ProjectX\bin-debug\ProjectX.html"/>
<stringAttribute key="com.adobe.flexbuilder.debug.ATTR_PROJECT" value="ProjectX"/>
<stringAttribute key="com.adobe.flexbuilder.debug.ATTR_RUN_URL" value="C:\Projects\ProjectX\bin-debug\ProjectX.html"/>
<booleanAttribute key="com.adobe.flexbuilder.debug.ATTR_USE_DEFAULT_URLS" value="false"/>
<listAttribute key="org.eclipse.debug.core.MAPPED_RESOURCE_PATHS">
<listEntry value="/ProjectX"/>
<listEntry value="/ProjectX/src/ProjectX.mxml"/>
<listAttribute key="org.eclipse.debug.core.MAPPED_RESOURCE_TYPES">
<listEntry value="4"/>
<listEntry value="1"/>

For any other properties, you can search for a relevant string in the .metadata folder of your 32-bit machine, then copy or create the appropriate file in your 64-bit system.  It’s a pain, but at least you’ll be able to get your work done.

December 10, 2008

How to use the REST API in Elgg 1.1

Filed under: Open Source — Tags: , , — admin @ 3:50 pm

The Elgg documentation describes the REST API rather superficially, yet provides no samples, and is now inaccurate for the recent versions of Elgg.

After a minor modification to your .htaccess file, using the REST API is very simple, and, in many cases, avoids the need to create individual files in your Elgg root directory to provide custom functionality.

First, let’s look at how your API method would be called:


Let me explain technically what’s going on first. Any API calls in Elgg are handled by a page handler, i.e., when the HTTP request begins with /pg/..., the .htaccess (containing mod_rewrite rules) tells Apache to use the Elgg file engine/handlers/pagehandler.php to handle the request. Elgg then determines which page handler to use, and invokes it. The REST API is known as an “API endpoint”. In this case, because the next part of the URL is rest, the API page handler passes control to the file services/api/rest.php.

The problem with the .htaccess that ships with Elgg is that it would cause the page handler to be invoked as pagehandler.php?handler=api&page=rest, discarding the querystring entirely. services/api/rest.php actually expects the method parameters to be in the querystring, so to get this to work, we simply change the following line in .htaccess:

RewriteRule ^pg\/([A-Za-z\_\-]+)\/(.*)$

to this:

RewriteRule ^pg\/([A-Za-z\_\-]+)\/(.*)$
engine/handlers/pagehandler.php?handler=$1&page=$2 [QSA]

Here is the code for a sample REST method, which would be implemented via a plugin. For this example, the following file is located at mod/rest_login/start.php (NB I had to put a space character before the ?php in line 1 for WordPress to render this listing):

  1. < ?php
  3. function rest_login_init() {
  4. expose_function('login', 'rest_login_handler', array(
  5. "username" => array("type" => "string"),
  6. "password" => array("type" => "string")
  7. ), "", "GET", false, true);
  8. }
  10. function rest_login_handler($username, $password) {
  11. $persistent = "1";
  12. $result = false;
  13. if (!empty($username) && !empty($password)) {
  14. if ($user = authenticate($username,$password)) {
  15. $result = login($user, $persistent);
  16. }
  17. }
  18. return $result;
  19. }
  21. // Add the plugin's init function to the system's init event
  22. register_elgg_event_handler('init','system','rest_login_init');
  24. ?>

The important differences between what is stated in the Elgg documentation and here are:

  • There is no register_method function. Instead, use expose_function as in line 4.
  • "type" must be specified for each parameter, as in lines 5-6.

Finally, here is an excerpt from engine/lib/api.php, which explains the parameters in more detail:

  1. /**
  2. * Expose an arbitrary function as an api call.
  3. *
  4. * Limitations: Currently can not expose functions which expect objects.
  5. *
  6. * @param string $method The api name to expose this as, eg "myapi.dosomething"
  7. * @param string $function Your function callback.
  8. * @param array $parameters Optional list of parameters in the same order as in your function, with optional parameters last.
  9. *       This array should be in the format
  10. *   "variable" = array (
  11. *                                       type => 'int' | 'bool' | 'float' | 'string' | 'array'
  12. *                                       required => true (default) | false
  13. *        )
  14. * @param string $description Optional human readable description of the function.
  15. * @param string $call_method Define what call method should be used for this function.
  16. * @param bool $require_auth_token Whether this requires a user authentication token or not (default is true).
  17. * @param bool $anonymous Can anonymous (non-authenticated in any way) users execute this call.
  18. * @return bool
  19. */
  20. function expose_function($method, $function, array $parameters = NULL, $description = "", $call_method = "GET", $require_auth_token = true, $anonymous = false)

There are limitations of the REST API, one of them being that the return value is rendered in a view. You can override the view by including the directory structure and file views/default/api/output.php in your module directory, but Elgg will still render it as HTML; if you need to process the result, you will need to parse this HTML. Also, if you decide to override this view, it will be overridden for any REST API method calls, not just the one implemented in your module.

One particular limitation of the sample above is that it does not authenticate! At least I couldn’t get it to work; the reason is that the API method is registered as not requiring a user authentication token. As a result, the method executes in unauthenticated fashion, and when authenticate() is called to attempt to authenticate the user, it always succeeds – regardless of the password – because pam_authenticate() returns true for the executing method. (Hopefully this would make sense to you if you’ve written an authentication plugin.)

September 30, 2008

Adding “Friends” access level to Elgg 1.1

Filed under: Open Source — Tags: , , — admin @ 10:43 am

Out of the box, Elgg 1.1 has three access levels: private, logged-in users, and public.

A few people have asked about how to add a further access level: friends. Currently, the only way to give friends access to an item is to create a friends collection, then specify that collection for the access level. The problem with this approach is that every you add or remove a friend, you need to update your collection manually.

The Elgg development philosophy is not to modify the core files, but instead to write plugins, if you need to modify any of the core functionality. However, in this case, there are no Elgg events that we can hook into, as access level is such a fundamental part of the core. Thus, we are forced to modify a couple of the core files.

Note that this won’t work for MySQL versions prior to 4.1, as they do not support subqueries.

Add line 147 in engine/lib/access.php:

  1. function get_access_sql_suffix($table_prefix = "")
  2. {
  3. global $CONFIG;

Change line 167 in engine/lib/access.php:

  1. if (empty($sql))
  2. $sql = " ({$table_prefix}access_id in {$access} or ({$table_prefix}access_id = 0 and {$table_prefix}owner_guid = $owner) or ({$table_prefix}access_id = -1 and ({$table_prefix}owner_guid = $owner or {$table_prefix}owner_guid in (select guid_two from {$CONFIG->dbprefix}entity_relationships er where er.guid_one = $owner and er.relationship = 'friend'))))";

Change line 201 to the following:

  1. $tmp_access_array = array(0 => elgg_echo("PRIVATE"), -1 => elgg_echo("FRIENDS"), 1 => elgg_echo("LOGGED_IN"), 2 => elgg_echo("PUBLIC"));

The last change in core is to add line 190 in languages/en.php:

  1. 'PRIVATE' => "Private",
  2. 'LOGGED_IN' => "Logged in users",
  3. 'PUBLIC' => "Public",
  4. 'FRIENDS' => "Friends",
  5. 'access' => "Access",

Finally, the pages module needs to respect the new access level, so modify mod/pages/start.php by adding lines 252-256, and 262-269:

  1. if (($write_permission!=0) && (in_array($write_permission,$list)))
  2. return true;
  4. if ($write_permission == -1) {
  5. $friends = get_friends($params['entity']);
  6. if (in_array($user->guid, $friends))
  7. return true;
  8. }
  10. }
  11. }
  12. }
  14. function get_friends($entity)
  15. {
  16. $friends = array();
  17. $entities = get_entities_from_relationship('friend', $entity->owner_guid, false, 'user', '', 0, '', 1000000);
  18. foreach ($entities as $friend)
  19. $friends[] = $friend->guid;
  20. return $friends;
  21. }

Powered by WordPress