Mark

Charts and graphs are an essential tool in modern web applications. The API we use for displaying charts and graphs in IP.Board presently is something we wrote in-house during IP.Board 2.x - it uses the PHP GD library to generate an image representing a chart. At the time, it was pretty amazing, but as times have changed a number of libraries for generating much more visually appealing and interactive graphs have emerged. For IPS Social Suite 4.0 we've decided to retire our GD-based charting library and embrace something a bit more modern and familiar for developers to work with. After looking at a number of different solutions, we decided to use Google Charts (although wrapped with a simple gateway PHP class). Google Charts look great, have great features, are commonly used (so we figure any third-party developers who want to add charts to their apps will be able to do so easily) and is a service provided for free with no API limitations. I'll show you an example of how simple it is to create a chart in IPS Social Suite 4.0. Let's look at the code I might use to show a graph of the number of members registered over time: /* Init Chart */ $chart = new IPSHelpersChart; /* Specify headers */ $chart->addHeader( "Date", 'date' ); $chart->addHeader( "Members", 'number' ); /* Add Rows */ $stmt = IPSDb::i()->build( array( 'select' => "COUNT(*) AS count, DATE_FORMAT( FROM_UNIXTIME( joined ), '%Y-%m-%d' ) as joined_date", 'from' => 'core_members', 'group' => 'joined_date', 'order' => 'joined DESC', ) ); $stmt->execute(); while ( $row = $stmt->fetch() ) { $chart->addRow( array( new IPSDateTime( $row['joined_date'] ), $row['count'] ) ); } /* Output */ IPSOutput::i()->output = $chart->render( 'LineChart', array( 'title' => "Registrations", ) ); As you can see, the code is extremely simple to understand and use. This is what the output looks like: By hovering over any point I'll see a tooltip with the value at that point. All of the chart types and options available in Google Charts are available to us. Let's make some changes to make the lines curved, get rid of the legend and show titles on each axis - I just change the last line of the code to: IPSOutput::i()->output = $chart->render( 'LineChart', array( 'curveType' => 'function', 'hAxis' => array( 'title' => 'Date', ), 'legend' => array( 'position' => 'none', ), 'title' => "Registrations", 'vAxis' => array( 'title' => 'Number of registrations', ) ) ); And now my chart looks like this:

To round up our previous blog entries on the post editor in IPS Social Suite 4.0, there's just a few extra features not previously mentioned to show off. @mentions @mentions are a common feature on social media sites like Twitter and Facebook. If you type an @ symbol and then start typing the name of a friend, an autocomplete menu shows so you can quickly then click on the user and they'll receive a notification that they've been mentioned. In 4.0 you can do exactly this to mention any user. Automatic Saving Currently, when you're typing a post, every 2 minutes the content of the post is saved, so that if you accidentally navigate away from the page, your post content can be recovered. The content is saved by making an AJAX request. In 4.0, we've rewritten this to use HTML5 web storage. This unloads this work to the browser, meaning no call needs to be made to the server. Because this is much more efficient, the save can be done much more frequently (every few seconds). This makes the autosave feature much more useful. In addition, we've expanded the feature to support attachments. So if you've uploaded files, these too will be automatically recovered. Essentially if you're in the middle of typing a post and you refresh the page, everything will reappear exactly as you left it. HTML Posting If you allow some users (like administrators) to post arbitrary HTML, they will see an additional "Source" button on the editor. When clicked, this will show them the raw HTML for the post and they can manipulate it here

Introduction Joining my previous entries about content and uploading features in post editor in IPS Social Suite 4.0, I'd like to take you through the customisation features on the editor. Toolbar layout The buttons that appear on the toolbar are completely customisable in 4.0 and you can set different layouts for desktop, tablet and mobile (so that you don't show more buttons than the device can show). This is what the management screen looks like: (This is an unfinished design - the tabs won't be be like that in the final version.) To move a button you just drag and drop. The buttons on the right allow you to add more rows or separators. Clicking on a button brings up a dialog where you can adjust where and to whom it shows: Adding Buttons There are two ways to add a button to the editor. The easiest way is to install a CKEditor plugin. CKEditor has loads of plugins, and installing is as easy as uploading the zip file from their site. Here's a screenshot of the symbol plugin being used: The second way is similar to how custom BBCode currently works, you specify the HTML code to be added when the user clicks on the button. Manually created buttons can optionally have a dialog popup to ask for an option. Design Just as you can install CKEditor plugins by uploading the zip file, you can do exactly the same with CKEditor skins to change the design of the editor. You then simply set for each skin on your community which CKEditor skin to use for it. BBCode Though no features in IPS4 insert BBCode-style tags into the editor (like is currently done for attachments, etc.) users can still type BBCode into the editor and it will work fine. We've rewritten how BBCode is parsed to be much more secure and reliable and produce more standards-compliant HTML (for those who are interested, it parses the post content into a DOM Document and examines only the text nodes for BBCode tags, then either splits the nodes surrounding it and inserts one for block-level elements, or wraps all subsequent text nodes in the appropriate formatting element until the end BBCode is found). The benefit to this is that there now no longer needs to be a "BBCode mode" - you can type BBCode straight into the editor, even complicated stuff like lists spanning multiple lines, and it comes out looking great. The downside to this approach is that custom BBCodes can no longer be added through the Admin CP. However, as mentioned above, we now have the ability to add custom buttons to the editor which work in a much more intuitive way, and can do everything that custom BBCodes could and more. For those who really want to be able to add the ability for custom BBCode, we've isolated the method that returns the supported BBCode (and information needed to parse them) into a specific method so that custom BBCode can be added with a very simple hook specific to that purpose. Conclusion There's still one more blog entry to go in our series on the editor. To finish up I'll be showing off some cool special features including how you can post using regular HTML.

Introduction In the last blog entry I introduced some of the features in the post editor in IPS Social Suite 4.0. In this blog entry I'd like to show you the uploading features in the editor. Using the "Image" and "Attachment" dialogs Along the bottom of the editor there are two buttons that deal with uploading files: image and attachments. Both present a dialog which looks like this: We decided to keep both an images and an attachments dialog as users wanting to insert an image will naturally look for the "Image" button - if however, you upload an image to the attachments dialog, it will work completely as expected. The upload panel here is based on HTML5 which supports drag and drop uploading, if your browser doesn't support this, it will use Flash, Silverlight or Google Gears if you have any of those installed, and if not it will fallback to a HTML4 & JavaScript implementation (none of these support drag and drop, but instead you click the "Choose Files" button just as you do now - the label in the box will change to reflect this). Uploaded files then show below the box (images will get a preview), and you can click on any to add them into the editor, or click the "Insert All" button. When you insert an attachment into the editor, it displays either the image if it's an image, or a link if it's anything else, just as it will actually appear in the post (rather than the current "[attach=XXX]" tag). You can also of course delete the attachment, which will automatically remove it from the editor if you've already inserted it. Video Demonstration Quick drag-and-drop In addition to interacting with the panels, if you're using a supported browser, you can drag and drop straight into the editor. It will automatically figure out whether the uploaded file(s) are images or other files and add them to the appropriate panel automatically. Video Demonstration Image URLs In the image panel, there is an additional "From URL" tab which allows you to insert an image from a URL, as you type the URL a preview is shown, and you can optionally link to the image. Video Demonstration My Files In IP.Board currently, there is a "My Media" button which allows you to insert content submitted either in other posts or elsewhere in the community (images in IP.Gallery or files in IP.Downloads for example) into the editor. In 4.0, this feature is found in the images and attachments dialogs. Just with normal attachments, the content is inserted as it will be shown rather than the current "[sharedmedia=XXX]" tag. Conclusion Please let us know what you think of the uploading features in the comments. Remember though that we're only half way through our series on the 4.0 editor. In my next blog entry I'll be talking about customising the editor and the place of BBCode.

Introduction The post editor is undoubtably one of the most frequently used features of the IPS Social Suite as it's the way users submit content to your community and functionality has evolved dramatically from the early days of forum software which consisted of a plain textbox in which users would type BBCode into the feature-rich WYSIWG (What You See Is What You Get) editors prevalent on the web today. For the IPS Social Suite 4.0 we really wanted to focus on making the editor as good as it can be: feature-rich, intuitive to users and highly customisable. Over the next 4 blog entries I'm going to cover the functionality of the editor and related features (attachments, emoticons, etc.) and the customisation options that will be available in 4.0. The theme of each blog entry will be: Here's a screenshot of what it looks like (by default, on a desktop, more on that later): Quotes A feature added to IP.Board in the 3.x series was "visual quotes" - when adding a quote the box shows directly in the editor as it will show in the post, rather than as just normal text with a quote BBCode wrapped around. In 4.0 we've rewritten how this works to use a CSS3 based solution rather than JavaScript to keep the citation header attached to the quote, which makes the feature much more reliable and easy to use when splitting quote boxes, moving the cursor before or after the quote box or dealing with embedded quotes. Code We've also rewritten how code can be inserted into the editor to be more reliable. Adding code now initially brings up an editor which supports syntax highlighting: And after inserting, you'll see the code exactly how it will be displayed after posting and you can interact with it exactly how you'd expect: Spoilers Spoilers are now also visual (though of course they're not true WYSIWYG as you need to be able to see what you're typing) and a built-in feature enabled by default. When you click the spoiler button you'll be given a darkened box in which to type the content: The spoilers display in the post as the same coloured box with a "click to view" message. When clicked, the dark box fades away revealing the content underneath. Emoticons We've completely overhauled emoticon management in 4.0. You can now create multiple "groups", drag and drop to reorder and quickly upload loads of emoticons at once. When you click the emoticon button in the editor, you'll see a popup with an overview of all the groups. If your community only uses the default emoticons, this is big enough to show all the emoticons, however, if you like to install lots of emoticons, you can select any group from the dropdown menu to show all emoticons in that group, or use the search bar to find a particular emoticon. In addition, your most recently used emoticons will show right at the top. Embedded Media In 3.0 we introduced a media BBCode tag. By wrapping a URL to media such as YouTube or Vimeo videos in media BBCode tags, the correct embed code would automatically be worked out and embedded into the post. In later versions, we added automatic embedding support, so just pasting the URL into the post would cause the video to be embedded. This still works exactly the same in 4.0 with a couple improvements: Previews If you type the URL to a supported media in the "Link" dialog, a preview will be shown in the dialog - you can choose to insert the embedded media or just a regular link. If you insert the embedded media, it will show directly in the editor. oEmbed Currently URLs which gets picked up for media embedding and the output they produce are managed in the Admin CP. The idea was that this would allow admins to add support for additional services, however, it has caused some problems when for example, YouTube has added new URL formats or changed their embed code and they don't work on most communities until we release an update to change the default record for YouTube. is an API which allows you to make a call to the provider giving them the URL and they respond with the best embed code to use, if that URL can be embedded. In 4.0, if a URL for YouTube, Flickr, Vimeo, College Humor or Hulu is inserted, we make a call to their oEmbed provider to obtain the embed code, so it's always up to date. Conclusion This is just the first in our series of blog entries on the editor in IPS4. Please do let us know what you think in the comments, but remember there's much more to come! [*]Content [*]Uploads [*]Customisation and BBCode [*]Special features Video Demonstration Video Demonstration Video Demonstration oEmbed

Introduction I previously wrote a blog entry about building tables in IPS Social Suite 4.0. Similar to tables, we also have trees. Trees in many ways look and behave similarly to tables, but can be distinguished mainly by the fact that trees show a collection of objects in a fixed order (often, though not always the order can be changed by the administrator) whereas tables can show data sorted however you like at the time. An example of trees would be the list of forums in IP.Board. Trees vary quite significantly in their individual implementations, for example: Objects in a tree often have parent/child relationships. Sometimes this relationship is between the same type of object (for example, forums in IP.Board or categories in IP.Downloads), sometimes the relationship is between different types of object (for example, applications and modules, where modules are always the child of applications) and sometimes it's a mix of both (for example, packages in IP.Nexus are always children of package groups, but package groups may or may not be children of each other). Objects in a tree can usually, but not always be reordered by the administrator. Objects in a tree can sometimes be enabled and disabled without being deleted, for example applications and modules. Sometimes individual objects can't when the rest can (for example, you can't disable the "Core" application). Trees usually have controls for adding, editing, assigning permissions, duplicating and deleting objects, though a subset of these controls, or additional controls may be available (for example, you can't edit the permissions on a package in IP.Nexus, but you can run a purchase report, which you can't do in any other trees). For IPS Social Suite 4.0, I wanted to create a central class to create trees - currently we duplicate a lot of functionality, and all our trees display differently (in some places, quite radically so), but without loosing any of the flexibility necessary given the various differences in each implementation. Because trees are both more complicated and flexible than tables, the method for creating one might seem complicated - however, when you compare it to having to write every part manually (including all the JavaScript) as was the case in IP.Board 3, you'll be shaving hours off of development time. There are two ways you can create trees in IPS Social Suite 4.0. The most common way is when each item in the table has a record in the database. There are occasions when this isn't the case (for example, the developer center displays trees created from JSON objects), and the Tree class can handle these, but in this blog entry I'm going to show you the more common method. Just like I did for tables, I'm going to take you through how I programmed a real-world example, specifically the tree for custom profile fields. Creating the classes Custom profile fields are arranged into groups, so I'm going to start by creating a tree that shows the groups. To do this, I need to create a class for the custom profile field groups, I'll set this class to extend IPSNodeModel which is an abstract class that provides most of the functionality we need. IPSNodeModel in turn extends a class called IPSPatternsActiveRecord which provides functionality to make a class an Active Record (by that I mean, an object of the class corresponds to a record in the database). In this class I need to define a few variables - I'll explain them below, but this is the code I'll write: /** * Custom Profile Field Group Node */ class _Group extends IPSNodeModel { /** * @brief [ActiveRecord] Multiton Store */ protected static $multitons; /** * @brief [ActiveRecord] Default Values */ protected static $defaultValues = NULL; /** * @brief [ActiveRecord] Database Table */ public static $databaseTable = 'core_pfields_groups'; /** * @brief [ActiveRecord] Database Prefix */ public static $databasePrefix = 'pf_group_'; /** * @brief [ActiveRecord] ID Database Column */ public static $databaseColumnId = 'id'; /** * @brief [Node] Node Title */ public static $nodeTitle = 'profile_fields'; } $multitons and $defaultValues are required by the IPSPatternsActiveRecord class. We don't need to do anything with them other than declare them. $databaseTable tells the IPSPatternsActiveRecord class what database table the records we need are in. $databasePrefix tells the IPSPatternsActiveRecord class the prefix used on all database columns. This isn't necessary, but since all the columns in my table start with "pf_group_", putting it here will save me typing it out every time. $databaseColumnId tells the IPSPatternsActiveRecord class which column contains the primary key. $nodeTitle tells the IPSNodeModel class what language string to use as the title on my page that shows the tree. Now I need to create a controller to display my tree - the developer center will set up the structure of this for me, then I just fill in the name of the class I just created: namespace IPScoremodulesadminmembersettings; /** * Profile Fields and Settings */ class _profiles extends IPSNodeController { /** * Node Class */ protected $nodeClass = 'IPScoreProfileFieldsGroup'; } Now I have a page which looks like this: Customising the rows You'll notice I now have two rows (because I have two rows in my database), but both are blank. This is because the IPSNodeModel class doesn't know what to use for the record title. Let's fix that by adding a method to our class: /** * [Node] Get Node Title * * @return string */ protected function get__title() { $key = "core_pfieldgroups_{$this->id}"; return IPSLang::i()->$key; } That code might seem a bit confusing, but note: $this->id gets the value of the "pf_group_id" column in the database. This is because the IPSPatternsActiveRecord class provides us with a __get method for retrieving the database row values. This is handy if we want to be able to modify the value returned for whatever reason, as we can override that method. We're retrieving the value for a language key rather than some kind of title field in the database. This is because in IPS Social Suite 4, data like this will be translatable, so if you have more than one language you can display different values depending on the user's language choice. So now we have this (I've clicked the dropdown arrow so you can see it's contents): Most of this is okay, but permissions aren't relevant for custom profile field groups, so let's get rid of that. The IPSNodeModel class has methods for checking if the user has permission to each button it displays (which by default check ACP restrictions, which I'll explain more about later) - we can simply override the method which checks for that button so it always returns false: /** * [Node] Does the currently logged in user have permission to edit permissions for this node? * * @return bool */ public function canManagePermissions() { return false; } And now it's gone: Making the buttons work The first buttons I need to make work are the add on the "root" row, and the edit on each row below that. I'm going to ignore the add button on each record row for now as that is for adding a child record and we haven't got to that yet - it will start working automatically when we add support for children. These buttons will display a form. In a previous blog entry I talked about our form helper class. I'm going to use this to build the add/edit form. To do this, I'll add two methods to my class to display the form and to save it's values - here they are: /** * [Node] Add/Edit Form * * @param IPSHelpersForm $form The form * @return void */ public function form( &$form ) { $form->add( new IPSHelpersFormTranslatable( 'pfield_group_title', NULL, TRUE, array( 'app' => 'core', 'key' => ( $this->id ? "core_pfieldgroups_{$this->id}" : NULL ) ) ) ); } /** * [Node] Save Add/Edit Form * * @param array $values Values from the form * @return void */ public function saveForm( $values ) { if ( !$this->id ) { $this->save(); } IPSLang::saveCustom( 'core', "core_pfieldgroups_{$this->id}", $values['pfield_group_title'] ); } Most of that should be self explanatory - however in the blog entry about forms I didn't mention the Translatable class. If you have one language installed, this will just display a normal text field, however, if you have more than one, it will show one for each language. It then returns an array, which we can pass to IPSLang::saveCustom() to save the values. This is what our form might look like if I have several languages installed: Or, more commonly, if I just have one: Now the add and edit forms are working, but since these are very small forms (they only have one input field) I'd like them to display in a modal popup rather than take the user to a new page (if the user has JavaScript disabled, a new page will do). To do this, I just add a property to my class: /** * @brief [Node] Show forms modally? */ public static $modalForms = TRUE; Next we have the copy and delete buttons. These will actually work by themselves (the central class will handle copying and deleting the records from the database), however, since we have translatable values, we need to make sure those too are copied and deleted appropriately. To do this, I'll override the two methods which handle copying and deleting: /** * [ActiveRecord] Duplicate * * @return void */ public function __clone() { $oldId = $this->id; parent::__clone(); IPSLang::saveCustom( 'core', "core_pfieldgroups_{$this->id}", IPSDb::i()->buildAndFetchAll( array( 'select' => '*', 'from' => 'core_sys_lang_words', 'where' => array( 'word_key=?', "core_pfieldgroups_{$oldId}" ) ), 'lang_id', 'word_custom' ) ); } /** * [ActiveRecord] Delete Record * * @return void */ public function delete() { parent::delete(); IPSLang::deleteCustom( 'core', 'core_pfieldgroups_' . $this->id ); } Search You'll notice that the system has automatically added a search box at the top of the table. In order to make this work, we need to add a simple search method: /** * Search * * @param string $column Column to search * @param string $query Search query * @param string|null $order Column to order by * @return array */ public static function search( $column, $query, $order ) { if ( $column === '_title' ) { $return = array(); foreach ( IPSLang::i()->searchCustom( 'core_pfieldgroups_', $query ) as $key => $value ) { try { $return[ $key ] = self::load( $key ); } catch ( Exception $e ) { } } return $return; } return parent::search( $column, $query, $order ); } Making the rows re-orderable The last thing I need to do to finish the handling of groups is make it so we can drag and drop to reorder them. To do this I just add another property to my class telling IPSPatternsActiveRecord which column contains the order ID: /** * @brief [Node] Order Database Column */ public static $databaseColumnOrder = 'order'; ACP Restrictions It's important of course to make sure we honour ACP restrictions. The easiest way to do this is to create individual ACP restrictions for add/edit/delete (this can be done in the developer center) with a common prefix, and then specify like so: /** * @brief [Node] ACP Restrictions */ protected static $restrictions = array( 'app' => 'core', 'module' => 'membersettings', 'prefix' => 'profilefieldgroups_', ); The system will now look for ACP restrictions with the keys "profilefieldgroups_add", "profilefieldgroups_edit" and "profilefieldgroups_delete" when performing those actions. Children Let's recap what we have so far with a video: http://screencast.com/t/TUBuuYBSBON Now that we have groups sorted, we're going to create another class for the actual fields which will show as children. The process is almost exactly the same as for groups. Since the process is the same, I won't go through the process step-by-step, but here is the class I've written if you're interested: Field.php The only changes are: I've declared two additional properties specifying the class name of the parent ("IPScoreProfileFieldsGroup") and which column contains the parent ID. I've declared an additional method to fetch an icon for the row so we can see what type of field this is. Just like we overwrote canManagePermissions for groups, I've also overridden canAdd in the same way, as you cannot add children to profile fields. Now all we need to do is link them up. To do this, I add a property to my group class telling IPSNodeModel the name of the class which contains children: /** * @brief [Node] Subnode class */ public static $subnodeClass = 'IPScoreProfileFieldsField'; The system will now automatically change the behaviour of our page in the following ways: When clicking on a group, it will expand out to show the fields under it. The search box will include fields as well as groups in its results. When clicking the "Add" button for a group, it will show the field to add a field to that group. When clicking the "Copy" button for a group, you'll have the option to copy children too or not. When clicking the "Delete" button for a group, you'll have the option to move children elsewhere or delete them too. (This is my favourite feature) In addition to being able to drag and drop fields to reorder, you can drag a field out of one group and into another. Here's a video: http://screencast.com/t/5fQwgle3EX

A few weeks ago, I posted a blog entry mentioning a new feature in 4.0 which aims to make development of applications within the IPS Social Suite (both for us and third party modification authors) easier. We focussed on managing the database schema in that blog entry and I'd now like to take you through the other features. Modules Two tabs (one for admin modules and one for front modules) allow you to view all modules and sections in your application. You can add modules (which will both insert it into the database and create the relevant files in the filesystem), change the default section for a module (which previously required a defaultSection.php file) and create new sections. When you're creating a new section, the form looks like this (this is for creating a section for an admin module): The "Type" field controls the code that will be placed in the file created for the section - the options are: "Blank" - which will create the class with no other logic, so the section will be blank. "Table" which will create a class with a boilerplate for displaying a table. "Node Controller" - which will create a boilerplate for displaying a tree of containers such as IP.Board forums, IP.Downloads categories, IP.Nexus groups, etc. We've not posted how this class works, but a future blog entry will give more details. No matter which type you select, the system will automatically generate a file, with a basic class structure already filled in, including ACP restrictions checks, etc. The "Menu Tab" field is admin specific and controls under which tab in the Admin CP the section should show (for example, we have some stuff from the "core" app under the "Look & Feel" tab). Previously, one would have to edit the menu.xml file for the module to add sections, and making sections appear under tabs other the default application tab was very difficult - the new system does it for you. The "ACP Restriction" field is also admin specific and allows you to select an existing ACP restriction which will control who can see the section. It also has a special "Create Restriction" option, which will cause the system to create a restriction, and use that. Previously, one would have to edit the permissions.xml file to create restrictions and then assign them in the menu.xml file. Of course - you can completely bypass this feature and manually create your module folders and section files, but the addition of this feature makes the process much quicker. Admin CP Menu This tab contains a graphical representation of the data which was previously stored in menu.xml files. Admin CP Restrictions This tab contains a graphical representation of the data which was previously stored in permissions.xml files. Extensions Extensions are ways in which applications interact with one another. Previously, you would drop extension files in your applications "/extensions" folder, though there wasn't much reasoning to the structure of the directory, it was difficult to know what extensions were available, and sometimes understanding an extensions requirements was difficult. In 4.0, the extensions directory is more structured - the format is owner app > extension type > extension file (so admin/group_form.php for example, is now core/GroupForm/*.php) so this tab provides a GUI for managing your applications extensions. Applications can also specify a boilerplate file for an extension, so you can see what extensions are available, and clicking the "add" button will create a file with a basic structure to get you going. Settings In 4.0, developers have much more control over how settings are presented, rather than all being dumped in the central "System Settings" table. With this, much of the data that was previously needed in settings.xml is no longer required, so we've simplified the process of creating setting to just providing a key and a default value. Versions The versions tab shows all of the application's versions and the database queries that the upgrader will run when upgrading to that version. It's sort of a combination of the versions.xml file and the setup folder. Queries are automatically added as you modify the database schema. Naturally you can also manually add queries, or specify a custom PHP script to run in that upgrade step.

Login Handlers are the different methods for logging into the IPS Social Suite. We currently support: "Internal", which is for accounts created natively through the suite. Facebook Twitter Microsoft (this is currently referred to as "Windows Live", though they rebranded to "Microsoft Account" a short while ago) LDAP "IPS Connect", which is our SSO solution for connecting your site with other IPS Social Suite installations or third-party applications. A generic handler for any MySQL database you have access to. In 4.0 we've made a number of changes to the Login Handlers which I wanted to mention. Improved Password Encryption We currently use a salted md5 hash for hashing passwords. md5 has been a popular password hashing technique for years - however, it is not the most secure hashing method. md5 is designed to be computationally efficient (meaning generating a hash is quick). The problem with this is that if a server were ever compromised to the point that someone were able to gain access to a database containing passwords hashed using md5, and someone were to use a program to generate and hash different strings repeatedly until a match were found, the password could be worked out. One particularly well-known program claims to be able to make 5.6 billion md5 hashes per second with a relatively modern GPU. Even with our hashing method which includes multi-level hashing and a salt, this means, assuming an 8-character long password using only alphanumeric characters were used, a password could be calculated in about 3 days. While I'm unaware of any cases of this actually happening, we want to make sure that our products are as secure as they can be. For this reason, in 4.0, we're migrating to Blowfish. Blowfish is a more cryptographically secure technique for generating hashes that is deliberately slow, meaning that even if your database were ever compromised, the passwords will still be secure. New Login Handlers In addition to the Login Handlers mentioned above, we've added support for Google and LinkedIn. Improved Facebook and Twitter support Currently, although you can log in with Facebook and Twitter, they're not treated on the back-end as true Login Handers. This is because of how Login Handlers in 3.x were designed (which was before such 3rd party login services were popular) in that they assumed you would provide a username (or email) and password directly into a form, and subsequently didn't accommodate the OAuth-style login processes. Since we've rewritten the way Login Handlers are designed, this means we can treat Facebook and Twitter (and Google and LinkedIn which both also use OAuth) exactly the same as the rest. Practically, this means you'll see Facebook and Twitter in the Login Handlers section of the Admin CP, and manage them as you would any other login method. Updated Microsoft Support Microsoft now support OAuth for login through them so we've updated to use that. In addition to being necessary for when they stop supporting the old way, it's much easier to set up for the administrator.

There's a table in the Admin CP of the IPS Social Suite that I really like - the members table. It has some really cool options - you can reorder the data just by clicking on a column head; you can quickly search for a member by typing a name into a search box at the top; there's some filter options to quickly show banned, locked, spam and validating members; and there's an advanced search form to search for members based on practically any criteria. It would be great if these features were available elsewhere. So much like we did for forms, we decided to create a central helper class for building tables. To demonstrate how it works, I'm going to go through, step by step, how I recreated the Admin CP members table in IPS 4. It starts with one line to create the table, and another to pass it to the output class: /* Create the table */ $table = new IPSHelpersTableDb( 'core_members', 'app=core&module=members&section=members' ); /* Display */ IPSOutput::i()->output = IPSOutput::i()->getTemplate( 'global' )->block( 'members', $table ); With just those two lines, you'll see this: Some things to note: We're calling IPSHelpersTableDb - the "Db" part indicates that the source of data for our table is a database table. There are other classes to use, for example, a JSON document as the data source. We pass it the name of our database table (or for the other classes, whatever the data source is) and the query string part of the URL where we're going to be displaying this (which we need to build the links and AJAX calls). I'm passing it to the output through a template called "block" which simply adds the dark-blue bar at the top, which isn't actually part of the table itself, and some padding. The "members" parameter is the key for the langauge string to use in that dark-blue bar. I'm passing $table directly to the template - the helper class has a __toString method which renders the table, so the output class thinks it's been given a normal string. The first obvious thing is that we're showing all the columns in the database table, which obviously we don't want. So let's add another line to specify which columns we want: $table->include = array( 'name', 'email', 'joined', 'member_group_id', 'ip_address' ); In this example, I'm giving the helper class a list of columns to include - I could alternatively pass a list of columns to exclude, if that would be more appropriate. The output is now this: Some things to note: It's worked out pagination itself. When you click a pagination link, the contents of the table will update with AJAX, including changing your browser's URL (unless you have JavaScript disabled of course, in which case it will work like a normal link). Pagination defaults to 25 results per page, but you can change that just by changing a property in the class. All the columns are clickable, which will resort the results. You can sort any column ascending or descending. Resorting will also update with AJAX (including changing your browser's URL), unless JavaScript is disabled. I want the headers to display something more meaningful than the column name. The system will automatically look for language strings which match the column name - you can also optionally specify a prefix, and it'll look for langauge strings which match that followed by the column name. Let's specify a prefix: $table->langPrefix = 'members_'; And I'll then create some language strings that match that (so "members_name", "members_email", etc.). The output is now this: Next - we need to change how we display some of those values. The joined date and the group are displaying the raw values from the database, but we want something more meaningful than that. To format the values, we simply create an array of lambda functions - one for each we want to format: $table->parsers = array( 'joined' => function( $val, $row ) { return IPSDateTime::ts( $val )->localeDate(); }, 'member_group_id' => function( $val, $row ) { return IPSMemberGroup::load( $val )->formattedName(); } ); I'm also going to add one additional line to specify the "main" column, which applies some additional styles: $table->mainColumn = 'name'; The output is now this: Some things to note: I'm using the IPSDateTime class to format the joined date. The ts method in this is a factory method which takes a UNIX timestamp and returns an object of IPSDateTime. IPSDateTime extends DateTime, so all the features of that class are available to us. The localeDate method returns a string with the date formatted appropriately according to user's locale. The IPSMemberGroup::load call being executed for each result may look like it might be resource intensive, but it caches objects it creates, so it's only actually "loading" each group once. Now I want to add a column with the user's photo. There isn't a single "photo" column in the database we can use for this (since the photo could be one they uploaded, a photo from their Facebook account if they're using Facebook Connect, a Gravatar image, or some other things), we need to use a method in the IPSMember class. This isn't a problem. I can simply add an element to our list of fields to include and add that into the parsers. $table->include = array( 'photo', 'name', 'email', 'joined', 'member_group_id', 'ip_address' ); $table->parsers = array( 'photo' => function( $val, $row ) { return IPSMember::constructFromData( $row )->photo('mini'); }, I'll also want to specify that we cannot use the photo column for sorting: $table->noSort = array( 'photo' ); The output is now this: Some things to note: Since this isn't a value which exists in the database, the value of $val in the lambda function will be NULL, however, $row has all the data for that record. We're not using IPSMember::load to get the member object, since that would execute an additional query for every result, which would be resource intensive, and unnecessary since we already have that data. Instead, we use the constructFromData method and pass it the row from the database. Next, I want to specify the default sorting. This is done with just two lines of code: $table->sortBy = $table->sortBy ?: 'joined'; $table->sortDirection = $table->sortDirection ?: 'desc'; The output is now this: Now, I want to add a quick search box. All we need to do is specify which column the quick search should look at: $table->quickSearch = 'name'; The output is now this: Some things to note: As you type, results are obtained with AJAX. You can page through your results (the number of pages will update automatically) and reorder your results by clicking the headers without loosing your search. I also want to allow more advanced search options - like to search by email address, or joined date. To do this, I create a new array: $table->advancedSearch = array( 'member_id' => IPSHelpersTableSEARCH_CONTAINS_TEXT, 'email' => IPSHelpersTableSEARCH_CONTAINS_TEXT, 'ip_address' => IPSHelpersTableSEARCH_CONTAINS_TEXT, 'member_group_id' => array( IPSHelpersTableSEARCH_SELECT, array( 'options' => $groups ), function( $val ) { return array( 'member_group_id=? OR ? IN(mgroup_others)', $val, $val ); } ), 'joined' => IPSHelpersTableSEARCH_DATE_RANGE, ); To explain what's going on here: The keys are the columns we're letting the user search on. The values are usually a constant indicating the type of search that is appropriate for that column. The member_group_id element is a bit more complicated. It has to specify an array of options (I've omitted the code to generate $groups in this snippet, but it'll be at the end of this blog entry), and, because we need to search both primary and secondary groups based on the value, there's a lambda function to get the proper WHERE clause for the query. Now, next to the quick search box there's a button which will bring up a modal popup (or just take you to a new page if JavaScript is disabled) which looks like this: Some things to note:The date entry boxes use the HTML5 date input type: If your browser doesn't support that, there's a JavaScript fallback: And if you're really awkward and are using a browser that doesn't support the HTML5 date input type and have JavaScript disabled, you'll see a regular text box where you can enter a date in practically any format, and it'll work it out. After performing the search, you can reorder your results by clicking the headers without loosing your search. Now, I want to add some filters so you can quickly see banned, spam, locked and validating members. To do this, you create an array simply specifying the WHERE clause to use in the query for each filter: /* Filters */ $table->filters = array( 'members_filter_banned' => 'member_banned=1', 'members_filter_locked' => 'failed_login_count>=' . (int) IPSSettings::i()->ipb_bruteforce_attempts, 'members_filter_spam' => '(members_bitoptions & ' . IPSMember::$bitOptions['bw_is_spammer'] . ') != 0', 'members_filter_validating' => 'v.lost_pass=0 AND v.vid IS NOT NULL' ); For this though, I'll also need to join the core_validating database table, so we add one more line for that: $table->joins = array( array( 'from' => array( 'core_validating' => 'v' ), 'where' => 'v.member_id=_0.member_id' ) ); The output is now this: Some things to note: The helper class will add the "All" filter automatically. It's getting the word to use for the filter by looking for a language string with the same key as the key in the array passed. Like everything else, clicking a filter updates the results with AJAX and the filter is retained in searches. Finally, the last thing I need to do is add a column with some buttons. You can specify a normal array for buttons to show in the header, and a lambda functions to return an array for buttons to show for each row: $table->rootButtons = array( 'add' => array( 'icon' => array( 'icons/add.png', 'core' ), 'title' => 'members_add', 'link' => 'app=members&module=members&section=members&do=add', ) ); $table->rowButtons = function( $row ) { return array( 'edit' => array( 'icon' => array( 'icons/edit.png', 'core' ), 'title' => 'edit', 'link' => 'app=members&module=members&section=members&do=edit&id=' . $row['member_id'], ), 'delete' => array( 'icon' => array( 'icons/delete.png', 'core' ), 'title' => 'delete', 'link' => 'app=members&module=members&section=members&do=delete&id=' . $row['member_id'], 'class' => 'delete', ), ); }; Our finished table looks like this: And behaves like this: http://screencast.com/t/KMFq8zCE To recap, here's the code, in it's entirety to generate that table: /* Create the table */ $table = new IPSHelpersTableDb( 'core_members', 'app=core&module=members&section=members' ); $table->langPrefix = 'members_'; /* Columns we need */ $table->include = array( 'photo', 'name', 'email', 'joined', 'member_group_id', 'ip_address' ); $table->mainColumn = 'name'; $table->noSort = array( 'photo' ); /* Default sort options */ $table->sortBy = $table->sortBy ?: 'joined'; $table->sortDirection = $table->sortDirection ?: 'desc'; /* Filters */ $table->joins = array( array( 'from' => array( 'core_validating' => 'v' ), 'where' => 'v.member_id=_0.member_id' ) ); $table->filters = array( 'members_filter_banned' => 'member_banned=1', 'members_filter_locked' => 'failed_login_count>=' . (int) IPSSettings::i()->ipb_bruteforce_attempts, /*@todo*/ 'members_filter_spam' => '(members_bitoptions & ' . IPSMember::$bitOptions['bw_is_spammer'] . ') != 0', 'members_filter_validating' => 'v.lost_pass=0 AND v.vid IS NOT NULL' ); /* Groups for advanced filter (need to do it this way because array_merge renumbers the result */ $groups = array( '' => 'any_group' ); foreach ( IPSMemberGroup::groups() as $k => $v ) { $groups[ $k ] = $v; } /* Search */ $table->quickSearch = 'name'; $table->advancedSearch = array( 'member_id' => IPSHelpersTableSEARCH_CONTAINS_TEXT, 'email' => IPSHelpersTableSEARCH_CONTAINS_TEXT, 'ip_address' => IPSHelpersTableSEARCH_CONTAINS_TEXT, 'member_group_id' => array( IPSHelpersTableSEARCH_SELECT, array( 'options' => $groups ), function( $val ) { return array( 'member_group_id=? OR ? IN(mgroup_others)', $val, $val ); } ), 'joined' => IPSHelpersTableSEARCH_DATE_RANGE, ); /* Custom parsers */ $table->parsers = array( 'photo' => function( $val, $row ) { return IPSMember::constructFromData( $row )->photo('mini'); }, 'joined' => function( $val, $row ) { return IPSDateTime::ts( $val )->localeDate(); }, 'member_group_id' => function( $val, $row ) { return IPSMemberGroup::load( $val )->formattedName(); } ); /* Specify the buttons */ $table->rootButtons = array( 'add' => array( 'icon' => array( 'icons/add.png', 'core' ), 'title' => 'members_add', 'link' => 'app=members&module=members&section=members&do=add', ) ); $table->rowButtons = function( $row ) { return array( 'edit' => array( 'icon' => array( 'icons/edit.png', 'core' ), 'title' => 'edit', 'link' => 'app=members&module=members&section=members&do=edit&id=' . $row['member_id'], ), 'delete' => array( 'icon' => array( 'icons/delete.png', 'core' ), 'title' => 'delete', 'link' => 'app=members&module=members&section=members&do=delete&id=' . $row['member_id'], 'class' => 'delete', ), ); }; /* Display */ IPSOutput::i()->output = IPSOutput::i()->getTemplate( 'global' )->block( 'members', $table );

When developing, modifying the database schema (such as adding a column to a table) can be surprisingly time consuming. Currently, we have to: Make the change locally Change the installer Add the query to make the change to the upgrader for whatever version we're working on Let the other developers know so they can run the query to make the change in their installs. This can cause issues, especially at the last step - we currently use a large .sql file in the trunk directory of our repository which we add queries to that we need everyone else to run - sometimes, one of us doesn't notice this has been updated. I'm sure also, third party modification authors are familiar with this annoyance. In 4.0, one of the things I really wanted to do was build a central "Developer Center" from which both us and modification authors can manage aspects of the application without digging into installer/upgrader files, XML files and manually copying and pasting things when creating a new module or extension. Part of this developer centre is a GUI for modifying the database schema. When you make changes, the changes will automatically be ran against your local database and added into the installer and upgrader. Internally, we'll also then have a special script which runs when we SVN update to copy these changes over when another developer makes a change. The first page in the Database Schema Management will be a list of tables: You can add a table to the list either by creating a new table, importing a table which is already in the database or uploading a .sql file containing a CREATE TABLE statement: When you edit a table, you can manage the columns, indexes and rows which are inserted by default: If you try to edit a table and your local database does not match what the schema has, you'll be shown the conflicts: This, plus the other features in the Developer Center, which we'll talk about in later blog entries, make the process of developing applications easier for both us, and third party authors. Fun fact: I'm writing this blog entry from the home of IPS in Lynchburg, VA rather than my usual office in the UK :smile:

Forms are an ubiquitous aspect of any web application. In the IPS Social Suite, particularly in the Admin CP, I often find myself copying and pasting code in various places to create a form. We've had the ipsRegistry::getClass('output')->formInput() and similar methods since 3.0, but you still have to copy all the HTML to display the rows, and write all the code to validate it yourself. Copying and pasting code is something all developers hate. It's a red flag that you're probably doing something wrong. In IPS 4.0, we've written a central form building helper class to alleviate this. Just as a reminder: Everything in this blog is a work in progress - naturally someone with a much keener design sense than I will be going over the interface - I'm just demonstrating the functionality, not a finished product :smile: The Basics Let's say I want to create a form with a single text input field. I simply initiate the form, add an object representing the input, and then display the form (it has a __toString method) - like so: $form = new IPSHelpersForm(); $form->add( new IPSHelpersFormText( 'name', 'default value' ) ); IPSOutput::i()->output .= $form; The helper will automatically look for language strings with the same name as the form element and use them. If there is a language string with the same name and then "_desc" - it'll use that as the description. So the above code, in the ACP, produces something like this: Required Let's say I want the field to be required - I just pass a third argument indicating so. When the form is submitted, if no value has been provided, it will automatically display an inline error: new IPSHelpersFormText( 'name', '', TRUE ) Options The fourth argument I can pass is an array with options dependent on the type of input field I'm adding. These options may change the way a field is displayed or add additional validation. So for a text field, I can specify the minimum and maximum length (which, naturally takes multibyte languages into consideration): new IPSHelpersFormText( 'name', '', TRUE, array( 'minLength' => 5 ) ) For a number field, I might specify the number of decimal points to round to, which it will do on the fly: new IPSHelpersFormNumber( 'name', 0, TRUE, array( 'decimals' => 2 ) ) Watch Video Or I could add an "Unlimited" checkbox, which is quite common for Admin CP settings: new IPSHelpersFormNumber( 'name', 0, TRUE, array( 'unlimited' => TRUE ) ) Custom Validation If the built-in options don't provide enough validation for a given need, you can pass a lambda function as a 5th argument: new IPSHelpersFormText( 'name', '', TRUE, array(), function( $val ) { if ( $val === 'Bad Value' ) { throw new IPSHelpersFormException( 'That value is not allowed.' ); } } ) Uploads Of course - simple input fields aren't all that can be done. How does drag and drop uploading sound? new IPSHelpersFormUpload( 'name', NULL, FALSE, array( 'multiple' => TRUE ) ) Watch Video (By the way, you'll notice a stripy bar flashes up briefly - this is a real progress bar, but because I'm uploading to localhost it's filling faster than it displays - in practice, you'll see a nice smooth-filling progress bar) Getting the values The form helper will by default (you can override it of course) submit to the same page it's on. You can check if it has been submitted (and passed validation) and obtain the values if so simply by calling the values() method - here's an example: $form = new IPSHelpersForm(); $form->add( ... ); if ( $values = $form->values() ) { // $values contains the values from the form } else { IPSOutput::i()->output .= $form; } The values will be returned in a way that is appropriate to the input type. For example, a text input field will return a string, number input will return either an integer or float, an upload field will return an IPSFile object (or array of them, if you're accepting multiple files), etc.

In 4.0, we have made changes to the database class to make use of prepared statements. For insert and update queries, the syntax is the same as it always has been: IPSDb::i()->insert( 'table', array( 'foo' => 'bar' ) ); However, where previously the database class would try to work out the type of variable passed to it - it now binds these to a prepared statement. The real usefulness of this change though, is apparent when you need to use a where clause. Where previously you'd have to do something like this: $this->DB->buildAndFetch( array( 'select' => '*', 'from' => 'table', 'where' => "foo='" . $this->DB->addSlashes( $foo ) . "'" ) ); You can now do: IPSDb::i()->buildAndFetch( array( 'select' => '*', 'from' => 'table', 'where' => array( 'foo=?', $foo ) ) ); We calculate the datatype based on the variable datatype, so previously where you had to do things like $this->DB->setDataType( 'foo', 'string' ) when you wanted to store a value like '01' - ew) - you can now just cast the variable to whatever datatype you like. For example, if you wanted to ensure that the variable was cast as a string to avoid issues where a user name of '007' was detected as an integer and converted to '7' then you'd use: IPSDb::i()->buildAndFetch( array( 'select' => '*', 'from' => 'table', 'where' => array( 'foo=?', (string) $foo ) ) ); Not only is this easier to type and to read, it ensures that the database class always takes care of escaping things properly.

A while back, we casually mentioned in a blog entry that 4.0 would be next major version after 3.4. Development of 4.0 is underway and we're going to be using this new blog to talk about development as we go. As Brandon mentioned a couple of days ago - the format of these entries is going to be developer-specific. If what we're saying doesn't make much sense right now, we will still be putting announcements up in our main blog when they're finished and ready for everyone to see. Because of that, it's also worth bearing in mind that everything is subject to change. I'm going to be posting code samples, screenshots and so forth - but everything in this blog is a work in progress - not the final product - and that will probably show. With that out the way - let's talk about 4.0! :D The file structure Currently, applications are mostly self contained in their folders (which is either /admin/applications, /admin/applications_addon/ips or /admin/applications_addon/other) however, other files are dotted around in /interface, /public, etc. In 4.0, applications will be completely self-contained within a single /applications directory. An application directory will look something like this: extensions dev css html img js lang admin front (it's "front" rather than "public" now) [*]interface [*]modules[*]setup [*]sources [*]tasks [*]xml You will notice the inclusion of a /dev folder. This will not actually be shipped in production, but rather replaces /cache/lang_cache/master_lang and so forth in the 3.x line. Outside of the applications directory, there will be a "system" directory, which contains core framework classes. Namespaces and autoloading In 4.0, we'll be making use of PHP namespaces and using an autoloader. The best way to demonstrate how this works is with a few examples:classDb (/ips_kernel/classDb.php) is now IPSDb and located in /system/Db/Db.php output (/admin/sources/classes/output/publicOutput.php) is now IPSOutput and located in /system/Output/Output.php class_forums (/admin/applications/forums/sources/classes/forums/class_forums.php) is now IPSforumsForum and located in /applications/forums/sources/Forum/Forum.php IPSDispatcherFront and IPSDispatcherAdmin are two new classes (with similar functionality to ipsController in 3.x) and both extend IPSDispatcherDispatcher - all 3 are located in /system/Dispatcher/ in individual files. Better framework design Where appropriate, classes are being refactored to make better use of appropriate design patterns. One lovely side-effect of this is ipsRegistry no longer exists. Instead of, for example ipsRegistry::DB() you now use IPSDb::i() - the Db class uses a multiton design pattern (I didn't pass any arguments in that example, which doing will cause the Db class to load conf_global.php and create the default database connection, but I could have passed it a key) - the i method in this case will create the database connection if it doesn't already exist. To give another example - IPSMember (the new IPSMember) uses an Active Record pattern. So there's no more of this: IPSMember::isInGroup( 1, 4 ); It's now, the much more logical: IPSMember::load( 1 )->isInGroup( 4 ); Monkey patching hooks One of the great things about the IPS Community Suite is hooks - you can easily create a class and instruct the framework to use that instead of a core class. Now, I don't know about you, but I really, really, really hate having to do this: $class = IPSLib::loadLibrary( '/path/to/file', 'myClass' ); $object = new $class; Especially if what you want to call is a static method, in which case it can't be done. You want to of course, just be able to do: $object = new myClass; or: myClass::myStaticMethod(); There is a concept in software engineering to do this sort of thing, called monkey patching, and by clever use of the autoloader, we've managed to make this work. loadLibary and loadActionOverloader are no more. These points are of course, just the beginning of 4.0. Stay tuned for more :smile:

The "URL to refer to" has caused confusion that it is the URL to give out

It is possible that the stock levels may decrease past 0 if multiple orders are received simultaneously.