XPP Proposed Requirements
$Revision: 1.1.1.2 $
$Date: 2000/07/24 22:31:59 $

Requirements Assertions & Questions Parser XML Support Assertions: We can not use the XML::Parser module to parse XML pages. The current XML parsers for perl (based on XML::Parser which uses the C expat library) will not parse the HTML in XPML pages, making it useless as the core parser of XPP. The standard syntax will look like: <XPP TAG KEY1="VALUE" KEY2="VALUE" /> For just parsing perl we'll use <XPP>PERL</XPP> For printing we'll use <XPR> PRINT </XPR> Questions: If we can not use the official XML parser, we should make sure our parser code is abstracted so that we could one day rewrite it in C or use the XML parser later. Writing the parser in C is an option, but using an XML parser doesn't seem likely. An XML parser will always have a problem with HTML that is used in XPML pages. The XPP Parser must be very fast. Assertions: The speed of execution is more imporatnt then the speed of parsing. XPML Compilation at the Apache Parent Level Assertions: There should be a way to tell XPP when it is loaded into Apache, what pages is should compile. These can then be 'shared' by all subprocs. All new features should be compatable with this approach. For instance, pre-compiling of code doesn't work (remember <?= ?> XPML Pages should be cached in memory Assertions: Each time an XPML or included XMI file is requested, XPP should check that files mtime to see if it has been modified and recompile it if it has. XPML pages will be cached in memory as objects that contain coderefs? Questions: There should be a way of turning off the 'stat'ing of files. Assertions: There should be a PerlSetVar flag to tell XPP to not check files for their modification time. This will dramatically speed requests up. This will require restarting the web server when changes to pages are made. Sharing would increase if stating was off, because cached, compiled pages would never need re-compiling, so would stay shared in the parent proc. Questions: Includes There needs to be a way of including dynamic AND static files Assertions there are xinclude (dynamic) and include (static) methods for including files. Includes will be relative to the XPPincroot. There should be the ability to include and xinclude URLs as well. Included files will be in their own scope. Questions How could you xinclude a URL? An external code source (i.e., a web server which serves XMIs without parsing?) There should be a way of including files from within the web root. Questions This is currently supported, but only be letting the user type system rooted paths. This seems like a security hole. A better system should be thought of. There needs to be a way of passing parameters between includes. Assertions The second to Nth paramater you can pass the xinclude method will be the other variables you want to pass to that template. There should be a way to force a named variable into the scope of an include? This would save XPML programmers from having to shift() variables into the include, allowing them to use variables that are 'already there.' Questions There should be a way to include a file within the same namespace. Assertions: There should be an option of copying the code into the current page's code. Another reason this wasn't done before was because there was no way of knowing when an xmi had changed (in order to reload it), if the code was just copied in verbatim. Each page object could contain an array of files (or other expiry means) which would invalidate the cache if any failed an mtime test. There should be an option of copying the code from an included file into the current page's code. When the page is included (in this special way) it can be put in an if block which checks that files mtime. Each page object could contain an array of files (or other expiry means) which would invalidate the cache if any failed an mtime test. Questions Is it possible to have the code in memory only once, but be scoped to whomever is calling it? Although the no scope; pragma has been suggested on p5p (a cool feature, I might add!), there is no way that I know of to do this currently. Even if the above isn't possible, there should still be an option of copying the code into the current page's code. Another reason this wasn't done before was because there was no way of knowing when an xmi had changed (in order to reload it), if the code was just copied in verbatim. I think this too can be solved with creative code. When the page is included (in this special way) it can be put in an if block which checks that files mtime. Each page object could contain an array of files (or other expiry means) which would invalidate the cache if any failed an mtime test. Would this require a two-pass parsing approach? I don't believe so. There should be a TAG (non-perl) syntax for doing includes and passing parameters to includes. SEE XPP TAGS Included files should be cached once per process, not once per include. Assertions: Each XPP xpml or xmi will be stored in global memory with a timestamp. Each time someone or a page requests another xpml object it should look in this global memory space.     XPP Tags There should be a way of executing of using TAGS instead of pure perl to get things done in XPP. Assertions: Tags need to be extensible, ie. There needs to be an easy way of adding tag functions. When a new tag is created, there should be a way of telling XPP what keynames are required for that tag. tags map to actual methods (which can overloaded). Project Specific modules for defining tags Questions: Ask XPML authors what they're common tasks are. Using TAGS should not slow down execution. Assertions The parser will just replace the tags with valid perl so there would be no difference using or not using the tags. Tags should be XML compliant. Assertions Even if we can not use the XML parser we should make the tags as XML compliant as possible for the future. <XPP TAG ARG1="VALUE" ARG2="VALUE" /> There still needs to be a way to embed Perl in tags. Assertions The XML compatable syntax used for TAGS will probably not be best for just embedding perl in. <XPP> CODE... </XPP> Questions There still needs to be a way of printing perl within tags. Assertions Same considerations as above. <XPR> PRINT </XPR> Print does not need a semicolon. Questions Query Param Tags Assertions Need tags for the following: Getting query params. Checking for the existence of Query Params. Testing the value of query params. Questions What other query operations should we make tags for? Do we really need tags? Query params are most useful inside PRINTs or conditionals. Should be just a built-in method. Loop Tags Assertions Need tags for the following: Looping through arrays Looping through closures (Carbon query objects) An easy way of displaying X of X objects on a page with next and previous buttons. Initialization of state data based on query object Data (Carbon) changes to handle starting at X Navigation output generated from initialized state data Checking to make sure an object is an object. There should be a way, with one set of opening and closing tags to, go itterate through a carbon query object, checking refs as it goes. (Wait for Carbon::query objects). Questions Restrict the number of iterations of a loop What other loop operations should we make tags for? Objects Assertions Need tags for the following: Calling methods against the APP object and getting their return. Printing methods (vars) of objects. <XPP OBJ="$a" ATTR="name" /> listing objects... There needs to be a standard way of getting and generating a list of objects, be it an HTML linked list, a scrolling list, a popup menu etc... Printing methods against the app object. Questions What other object operations should we make tags for? Other Assertions Need tags for the following: FLOW CONTROL if statements... (not sure about this one) creating vars, setting vars (???) Shifting variables in (at the top of xmi's) xincludes iterative xinclude (given a list and an xinclude it will xinclude once for each item in the list) <XPP INCLOOP="inc.xmi" ON="$query" /> Questions What other operations should we make tags for? Comment Assertions Comment tag.. tag should have opening and closing which says anything in between should be ignored by the parser. This is important when commenting out code so the code doesn't show up in the outputted HTML. <XPPCOMMENT> ... </XPPCOMMENT> This will need to balanace tag start and end characters (< and >), and end on a balanced XPPCOMMENT tag. This will allow for commenting of code that zero or more XPPCOMMENT tags. Form Assertions Need tags for the following: INPUT Tags SELECT Tags Begin/End Form XPML authors should be able to send other key value pairs to form tags which just get passed through to the HTML. There needs to be a way for the XPML author to specify standard validation information per form field. They should be able to specify multiple validation params per form field. XPP Form tags should know to populate themselves with whats in the query object unless overridden. We should use the HTML_* methods from Chromium to make creating these dynamic forms easier. Should we move those HTM_ methods to a library outside of Chromium to be used by all the modules. For SELECT loops there should be an easy way of passing an array or Carbon closure or query object to a SELECT loop to have it generated for them. <XPP SELECT MULTIPLE="yes" SIZE="5" TITLE="Choose One... Please" FROM="$query" NAME="title" VALUE="id" /> For VALIDATION there should be some standard validation items the XPML author can use, like ISDATE or ISNOTNULL, but there needs to be a flexible system for adding VALIDATION CONSTRAINTS. The XPML author should NOT be assumed to know these things though, so there needs to be a way for XPP, through introspection into the Display objects to gain information about how a certain field wants itself to be validated. XPP could only do the validation itself using JavaScript since it won't know where they are submitting to. The above makes me believe we should standardize the get_ and list_objects methods. XPP should be able to look through the document to find out what 'vars' of each object they will be printing and make sure the correct vars are retrieved in the select for that object. A one pass parsing phase would collect information on which fields will be used, and in post-parsing modify the fields through a reference to the fields array which resides in the XPP page object's closure. Questions What other form tags should we make xpml tags for? Is there a way we could have the lazy XPML author surround an entire form with begining and ending XPML tags and have XPP assume each form tag in between is a dynamic form tag. IE. assuming there was an <XPP input...> tag we would let the XPML author do <XPP FORMBEGIN> <INPUT TYPE...></FORMEND> and it would assume or translate that <INPUT into <XPP INPUT> Filter Tags Assertions: The idea is to be able to have <FILTERTAG>content</FILTERTAG> and you could somehow change whats between the tags with the filter.... Is it too crazy? Caching XPP should be able to cache content in a flexible manner. Assertions: There should be a DBI/DBD model for choosing how the data is stored, retrieved, and deemed to be expired. The following suggested Store's should be supported File dbFile Memory DB Caching needs it's own table to associate caches (name/group paris) to the appropriate Store and expiry Most expiries would need only the name and group of the cache. Other information might be needed as implementations of new expiries are added. By using a cache tag in which the author would specify store and expiry types, the caching code in XPP should be able to handle everything. <XPP CACHE NAME="topheader" GROUP="nav" STORE="DB" EXPIRE="time" TIME="5m"> ... </XPP CACHE> Questions There needs to be a flexible way for checking to see if a cache has expired. Assertions: XPP::Cache would inherently create, modify, and remove both Store and Expiry entities as necessary (both Store and Expiry parameters would be passed to XPP::Cache methods). The following suggested Expiry's should be supported: Time Duration database flag database timestamp file non-existant file date old Questions: There needs to be a simple way of populating and re-populated named caches. Assertions: XPP::Cache table rows could have a flag for a cache-transaction in progress. Notes When a cache is deemed to be expired, someone has to repopulate it. In an ideal world, the first person to see that it was expired would begin a 'transaction' somehow to repopulate it... subsequent requests that occur while the first request is being processed will see that its expired, see that they can't repopulate it because it is locked and just generate the dynamic content themselves. Once the process repopulating the cache is done, it has to reset the expiry and unlock the cache. Obviously this is easier to do with database caches then other caches. It leads me to believe we need to have a suggested Store and Expiry architecture that has to deal with issues such as repopulating and locking. There needs to be a flexible way of deciding WHAT to cache. Assertions Questions Possible places to specify what to cache: Top of page to be cached. In xinclude surounding content. Ideally you would be able to define the cache in an xinclude tag, or maybe just at the top of an xincluded page. It would also be nice to be able to enclose text and perl on a given XPML page with other tags which tell it to cache whats in between, like (bad example)... <?xpp start_cach(options)?> stuff, stuff, <?=morestuff?> <?xpp end_cache(options)?>     Debugging Script Checker Assertions: There should be a script checker which can go through pages and look for common errors. Common errors include, no ref checking, global vars, etc.. Source Viewer Assertions: Ability to see the source code for an entire XPML with its XMIs. Line Numbers Questions: There should be a way of knowing what true line number an error occured on. Having errors report accurate line numbers is impossible. Clean Errors   Trace Mode   Other Sub-request support Questions Possibly for Java Servlet Support. This might sound completely insane, but I think there is a way to 'include' java servlets using Apache. That would be an pretty usefull tool.