Posts Tagged ‘architecture’

The Common Baseplane Method to Caching — memcached

Tuesday, May 27th, 2008

If you have ever worked on a massively high trafficked website, you know that cache is very important to keeping the server count down and being a superhero to your database servers. Cache can be bad and overly optimized but when you hit a certain threshold, relational databases, databases that are dimension modelled for data warehouse, and even server resources get exhausted. At that point you have two options, buy more servers, or more likely, cache read data.

Each platform has their own way to do this, but there is a common baseplane way to do caching, yes even in .NET. That is with memcached. Memcached is a very common and useful tool that makes caching data and cache layers in an application something that can be the same on every platform. The benefit of using memcached is it is open, common and it has APIs for nearly every popular web development platform (and can be wired in easily to platforms that don’t have their own cache mechanism). Why write your caching layer specific to a certain platform when you can memcache?

If you write high performance web apps and don’t memcache, I feel bad for your server budget and your late nights when that ad buy hits or something popular on your site becomes all the rave.

Perl API

An object-oriented Perl module can be found on CPAN as Cache::Memcached or in Subversion (ChangeLog). (GPL/Artistic)

The Perl API takes advantage of the server’s opaque flag support and sets its “complex” flag whenever the object being stored or retrieved isn’t a plain scalar. In that case, the Storable module is used to freeze and thaw the value automatically going in and out of the memcached.

There is also Cache::Memcached::Fast—another Perl client written in C, largely compatible with the original Cache::Memcached. Available on CPAN at http://search.cpan.org/dist/Cache-Memcached-Fast/.

PHP API

There are tons of PHP libraries available, in different conditions. But it now seems there’s an official one:

Python API

The Python client we’d previously released was just a prototype, and we don’t have regular Python programmers on hand. The folks at Tummy.com have took over maintenance. See ftp://ftp.tummy.com/pub/python-memcached/ for the latest versions.

Ruby API

Java API

A Java API is maintained by Greg Whalin from Meetup.com. You can find that library here:

An improved Java API maintained by Dustin Sallings is also available. Aggressively optimised, ability to run async, supports binary protocol, etc. See site for details:

C# API

There are multiple C# APIs:

C API

Multiple C libraries for memcached exist:

  • apr_memcache by Paul Querna; Apache Software License version 2.0
  • libmemcached by Brian Aker; BSD license. This is a new library, under heavy development.
  • libmemcache by Sean Chittenden; BSD license. This is the original C library. It is no longer under active development. You should try libmemcached instead.

Postgres API

The pgmemcache project allows you to access memcache servers from Postgresql Stored Procedures and Triggers. More details and downloads are available at:

Chicken Scheme

Lua

MySQL API

The memcache_engine allows memcache to work as a storage engine to MySQL. This means that you can SELECT/UPDATE/INSERTE/DELETE from it as though it is a table in MySQL.

A set of MySQL UDFs (user defined functions) to work with memcached using libmemcached.

Protocol

To write a new client, check out the protocol docs. Be aware that the most important part of the client is the hashing across multiple servers, based on the key, or an optional caller-provided hashing value. Feel free to join the mailing list (or mail me directly) for help, inclusion in Subversion, and/or a link to your client from this site.

The best part, they support all good platforms and even Lua, and wisely they left out VB.NET, no worries, VB.NET’ers will never know. Only kidding…

Finally, memcached is distributed, most cache layers included with platforms listed above are in process and per machine.  If you are running your code on a webfarm memcached is the only way to go.

Baseplane Tool: Is PureMVC the Cross Platform MVC Toolkit You Have Been Looking For?

Saturday, March 29th, 2008

PureMVC is quite a versatile MVC kit.  With implementations for AS3, .NET (c#), Python, PHP, Silverlight and other platforms it is quite a system and domain to spread that far and have consistency.  There are small changes but for the most post the MVC is the same structure across the platforms.  This can be very beneficial for a service firm or for a product base that needs to support many different platforms.

PureMVC is a lightweight framework  for creating applications based
upon the classic Model-View-Controller design meta-pattern.
This free, open-source framework is implemented in ActionScript 2 and
3, Java, C# and a number of other popular programming languages.
This allows development on a wide variety of platforms including:

  • Mobile Environments: FlashLite, .NET Compact Framework, J2ME
  • Server Environments: ColdFusion, J2EE, PHP, Python
  • Browser Environments: Flash/Flex, JavaFX, Silverlight
  • Desktop Environments: .NET, AIR, FLASH, J2SE

For Flex PureMVC happens to be my favorite MVC kit.  I only use one if absolutely necessary but PureMVC keeps it clean.  The great thing is that is works with or without Flex unlike Cairngorm and it is always up to date.  It is just an added bonus that is spans so many other platforms. There are a few things I don’t like about it in other platforms like the url naming but it is much better than kits out there now and Microsoft’s ASP.NET MVC most likely wont’ be cross platform *wink*.

Some info on the PureMVC framework (caution PDF):

PureMVC Manifold

Ports

Big O Notation in Design Theory

Saturday, March 22nd, 2008

Big O Notation is based on complexity theory and is something engineers and architects should know about do determine complexity and orders of magnitude in their data and scalability formal blueprints. Whenever you use any algorithm or port a formal function into code, math and reducing the orders of magnitude is what separates the fast from really fast.

Optimization can be evil, but solid base starting points are desired. Many times formal knowledge can be as needed as logical or physical separation and understanding service and standards format layering in your applications for the best evolution and versioning as well as performance. Formal engineering is what is separating companies like Google from the pack. Do you do formal?

Orders of common functions

Here is a list of classes of functions that are commonly encountered when analyzing algorithms. All of these are as n increases to infinity. The slower-growing functions are listed first. c is an arbitrary constant.

Notation Name Example
\mathcal{O}\left(1\right) constant Determining if a number is even or odd
\mathcal{O}\left(\alpha(n)\right) inverse Ackermann Amortized time per operation when using a disjoint-set (union-find) data structure
\mathcal{O}\left(\log^* n\right) iterated logarithmic The find algorithm of Hopcroft and Ullman on a disjoint set
\mathcal{O}\left(\log n\right) logarithmic Finding an item in a sorted list with the binary search algorithm
\mathcal{O}\left(\left(\log n\right)^c\right) polylogarithmic Deciding if n is prime with the AKS primality test
\mathcal{O}\left({n^c}\right), 0<c<1 fractional power searching in a kd-tree
\mathcal{O}\left(n\right) linear Finding an item in an unsorted list
\mathcal{O}\left(n\log n\right) linearithmic, loglinear, or quasilinear Sorting a list with heapsort, computing a FFT
\mathcal{O}\left({n^2}\right) quadratic Sorting a list with insertion sort, computing a DFT
\mathcal{O}\left({n^c}\right), c>1 polynomial, sometimes called algebraic Finding the shortest path on a weighted digraph with the Floyd-Warshall algorithm
\mathcal{O}\left({c^n}\right) exponential, sometimes called geometric Finding the (exact) solution to the traveling salesman problem (under the assumption that P ≠ NP)
\mathcal{O}\left(n!\right) factorial, sometimes called combinatorial Determining if two logical statements are equivalent[1], traveling salesman problem, or any other NP-complete problem via brute-force search, finding the determinant of a matrix with expansion by minors
\mathcal{O}\left({n^n}\right) n to the n  
\mathcal{O}\left(c_1^{c_2^n}\right) double exponential Finding a complete set of associative-commutative unifiers[2]

Not as common, but even larger growth is possible, such as the single-valued version of the Ackermann function, A(n,n). Conversely, extremely slowly-growing functions such as the inverse of this function, often denoted α(n), are possible. Although unbounded, these functions are often regarded as being constant factors for all practical purposes.

JSON and JSON-RPC

Saturday, March 22nd, 2008

JSON and JSON-RPC is one of the newest market baseplane and cross platform technologies.  It is very compact and based on minimal service exchanges with unique path and requests.  JSON the format is the result of bulky and bloated XML complaints.

Code Generation != ORM, Code Generation IS for the Code Generation

Friday, February 22nd, 2008

Code generation is not ORM (Object Relational Mapping). ORM is not code generation.

These are two separate ideas. ORM is part-of code generation but code generation is super to ORM.

I get in discussions with people that are strongly against ORM mostly due in part to Jeff Atwood’s post that ORM is vietnam. Yes it can be, and ORM is only for highly tailored teams and projects and creates bulk and uncontrollable code bases. BUT, code generation is valuable and here to stay.

Code generation can be a templating system that outputs code that exactly is the way you would write it and generate unit tests, db field mappings, basic skeletal architecture, maybe a data layer that accesses tables and views or any one of those things from a custom, open source, standard or commercial templating system.

Code generation can also be macros, db migration processes (SSPI or DTS or PL-SQL or n), adding a new file in yoru IDE (VS.NET/Eclipse/FD templates), configuration files, refactoring tools, and many other things. Code generation is a part of life for a programmer or engineer. In fact if you dont’ start getting a handle on the code generation systems that are evolving then you will be that much more without a solution when platforms advance.

Code generation can also be team building because many times people against code generation are hand coders. This is fine and this works great for one programmer. When you add programmer++ to any single team all of a sudden “styles” come into play. Code generation and basic framework or API code structures should be similar in a team environment so that others have a base layer of knowledge they can get up to speed on. Code generation brings the style battles and architecture to the forefront on how code should look in an organization. As you can imagine when you give 50 architects or developers the lead, you will get 50 different was of doing things, some good, some bad. Code generation ensures at least a basic structure of code and apis that can be used. Team integration is hard, code generation usually goes hand in hand with this if it is as small as new file templates in an IDE or a macro all the way to entire api framework generators.

It is important that code within the same project look and act the same. Same styles, same conventions, same architecture. One way of settling this is having code generation that everyone agree upon that can help at least structure projects the same, even if implementation and behind the API layer things are like the wild wild west.

Code Generation > ORM

Code Generation is usually guilty by association. We are hoping to change the perception, but again code generation is really only needed when you get beyond the single programmer mindset, or when you start to make a product family or open source.

Code Generation != ORM, Code Generation IS for the Code Generation



baseplane - technology platforms is proudly powered by WordPress
Entries (RSS) and Comments (RSS).

© 2006-2008 Ryan Christensen - template by drawk }}



Unless othewise specified the content in this site is licensed under a Creative Commons License
Your Ad Here Your Ad Here Your Ad Here Your Ad Here