Blogging from Project 2061 technology group
With a large set of test packets in need of transcription, little time to do so, and even less desire to spend the money on temporary workers, I needed to think of some other way to perform the task. Luckily I was able to come up with a method using a tool from Amazon that proved relatively easy to implement. Mechanical Turk .
Elements with opacity styling are stacked independent of their containing object. As a result, any positioned elements that do not have a z-index declaration and that appear in the HTML code prior to an opacity-styled element will appear underneath the opacity-styled element (see example). This is the defined behavior in the CSS specification, though currently only gecko- and WebKit-based browsers (Firefox, Safari, Chrome) exhibit the “quirk” in question. From the CSS3 Working Draft:
… implementations must create a new stacking context for any element with opacity less than 1.
I was trying to find a way to update some rows in a MySQL table where the condition for performing the update was based on a column in another table. Normally this isn’t an issue, just run UPDATE table ... WHERE key_column IN (SELECT key_column FROM ...).
I was having some problems, however, because the table that needed to be updated uses a mutli-column primary key. Normally I would select the relevant rows using a JOIN. Unfortunately, you can’t update the results of a JOIN. How then to select the relevant rows?
Turns out it’s easier than I thought. You can still use the WHERE ... IN clause, you just specify all of the columns that satisfy the condition: UPDATE table ... WHERE (key_column1, key_column2) IN (SELECT key_column1, key_column2 FROM ...).
I did not see any information about this functionality in the MySQL Reference Manual (1, 2).
A note way past due.
I updated the log format for flora.p2061.org (but not the other sites on flora). Some information is not included in the default “combined” format that could help with debugging (such as user cookies). I set up a new format that is similar to the IIS W3C extended format. Two values not included that could prove useful are bytes received (%I) and bytes sent (%O). The logio_module has to be enabled to record this information. It would probably be worthwhile to add this functionality once we start placing more public content on the server. For now the following format suffices:
#Fields: date time c-ip cs-host cs-version cs-method cs-uri-stem cs-uri-query cs(Referer) cs(User-Agent) cs(Cookie) sc-status time-taken
LogFormat "%{%Y-%m-%d %H:%M:%S}t %h %V %H %m %U %q %{Referer}i %{User-Agent}i %{Cookie}i %>s %D" webapp
I needed to do a mass resampling of around 280,000 images. There are a number of ways to do this, but I settled on doing it via PHP because the images are stored on our web server, the total size of the images is large (~10GB), and I didn’t want to kill my machine trying to get it done.
PHP is ideal for a task such as this: parsing directories and subdirectories for images is easy; resampling using the built-in library (GD) is a breeze; specifying the destination as a subdirectory is simple. The one minus was processor usage. Performing image manipulation eats up the CPU in a big way.
Luckily linux systems have a built-in utility for addressing a situation like this: nice. nice will “run a program with modified scheduling priority.” I’m running the image manipulation script using the following command:
nice --adjustment=19 php script.php
If nothing else is going on the script will use whatever resources are available. When anything with a higher priority executes, that program will take precedence over the script with regard to system resources. The script should thus not affect the responsiveness of the web server. This is the reason I was searching for this kind of functionality.
After creating a new structure for the current year’s piloting data there has been a bit of disconnect in development between the two data sets. There’s just not enough time to ensure that everything works across both data sets. There were basically two choices to moving forward: 1) construct a view of the new data structure that mimics the old data structure; 2) port the old data structure to the new one and continue redevelopment of the scripts. I chose the latter, mainly because there are some improvements I’d like to make to the interface in the process.
I created a series of SQL statements to run in MySQL that will convert the data from the packet_item_records and miscon_packet_refs tables to the packet_students, packet_data, and miscon_packetdata_refs tables. After the conversion I updated any pages that referenced the old data structure. So far in testing the data seems to have converted perfectly.
Minus one issue. Multiple selections for the answer choice questions (A, B, C, D) from 2006 were recorded as a generic Multiple rather than Y+NS, N+NS, etc. This value is not represented in the updated data format or on the data entry forms or summary tables. Rather then spend too much time addressing this issue I’m going to leave these values empty for now. I don’t expect this to be a problem since the researchers are focused on data for the current pilot and field tests. Also, I’m keeping the old version of the piloting data and scripts that interact with it online in case it’s needed.
I was tasked with creating a demonstration of a dynamically generated graph for one of the grant proposals being worked on at present. This isn’t something we’ve really done before in PHP, but I had a feeling it would not be a unique requirement. A quick search revealed that a PEAR package exists for just this purpose: Image_Graph.
Installation was not too difficult, though I did have to ensure that a few dependencies were installed before PEAR would load the package. Documentation for the module is not that great, but there are a number of samples and a fairly active discussion forum.
I did not find the graphing object very intuitive. It seems to me that a lot of the properties and functions are abstracted from the objects they affect. I suspect this was a design decision to allow for greater flexibility, but it may also be due to an older coding style. The current branch appears to have been developed in earnest starting in 2005, and it’s been a few years since a new release. Even so, my initial exposure leads me to believe the package may be stable enough for even a production-level project so long as sufficient QA has been performed.
It took me only a couple of days to hack together the demonstration. Though the requirements of the script were few, I decided to push the limits of my object-oriented PHP and create the graphing component as a class. Overall it was a good first-try at producing something of this nature. You can find the demonstrator at http://flora.p2061.org/weather_data/.
Piloting packets for this year are starting to come in and the data from these packets needs to be entered into the items utility. Because the questions in the packet have changed from previous years, and due to the rigid structure of the piloting data entry form and data tables, I decided to rewrite the data entry form. The new format allows more flexible on the front-end without requiring modification of the back-end. In addition I modified the way PHP interacts with the packet data from MySQL by utilizing a structured array to hold the data.
I’m currently cleaning up the items utility so that it can be brought into subversion. I’ve already deleted unused files and done some reorganization of the site files. Next up I’m going to work on some interface modifications that I hope will simplify the it as well as create some common navigational elements across pages. The final step will involve unifying the code in a control structure similar to other recent projects.
Because this is more of a long-term project it is being performed along side any current development of the items utility. As a result any modifications to the items utility should be brought to my attention so that they can be ported to the SVN working copy if necessary.
I recently wanted to update our install of WordPress to the latest version. WordPress is a fairly easy install, and we could learn a thing or two about application set-up by examining their code. But I recently switched to using subversion to deploy and maintain our install. In just the little bit I’ve used subversion so far, I believe development and deployment of our internal applications would be simplified by employing it for all our projects. Here’s a quick outline of the process, with examples based on my WordPress deployment.
« Previous Page — « Previous entries « Previous Page · Next Page » Next entries » — Next Page »