胺`X 公坙公坙 公犂公犂 公坙公坙 nj.公{f公犂  & ,cacheXhotliststatsnZhzArial 梗9公xps狶ANDwinWhat is the RCS Aggregator Cache? With this tool, the Radio Community Server can act as a RSS Cache for Radio Userland Aggregators. The RCS is able to gather the most popular RSS feeds on behalf of an entire community of Radio Userland Users. The Users Aggregators request those feeds from the RCS, rather than directly from the source. Content providers should receive significantly less requests of RSS feeds, resulting in less drain of bandwith and server resources. At present, this problem isn't significant, but as the usage of Aggregation greatly increases, some steps will need to be taken to preserve an efficient architecture This is a fully functional experiment for the Radio Userland platform. This functionality can be duplicated in other implementations. Please send any comments, suggestions, bug reports to mikel_maron@yahoo.com. Have fun! Install on RCS Requirements: You must have the Radio Community Server installed, and RSS Hotlist enabled. Download rcsAggregatorCache.root to you Radio Tools directory, and restart Radio. Enable the RCS cache on the rcsAggregatorCache prefs page. The RSS Cache is now enabled. Have members of your community install the tool, according to the instructions below. Install in Radio Requirements: Radio Userland, and an RCS that supports RSS Caching Download rcsAggregatorCache.root to you Radio Tools directory, and restart Radio. Enable the caching client on the rcsAggregatorCache prefs page. If your RCS does not support RSS Caching, please contact them. The Client Cache is now enabled. Aggregator requests will check with the cache before requesting directly from the source. How does it work This tool has two portions, one for the RCS and one for the client. The RCS maintains a list of the 100 most popular subscriptions of its users. When RSS Caching is enabled, periodically (presently 4 times an hour) the RCS requests and stores these feeds. It only stores the raw XML, and does not process them. A GET interface is provided to clients to request the cached feeds. For the Radio client, when the tool is enabled, a modified version of xml.aggregator.everyMinute is placed in the scheduler. This modified version will request and process the RSS Hotlist from the RCS, then run as a normal aggregator. Except, if a user's subscription is included in the Hotlist, the RCS Cache is queried for the feed. If that fails, the request is made directly from the provider. Future Directions There's lots of potential in the RCS. My next idea is to build a Hotlist of read articles, a sort of blogdex for actual reading habits. Development Notes Todos next version modified table - track whether original routines have changed, and notify author when that happens Jake says this will work local (adrobject = @addressOfTheThingToCheck); local (adrmyversion = @addressOfMyBackupCopy); if string (adrobject^) != string (adrmyversion^) { //do your thing here } uninstall rcsAggregatorSuite.background.everyMinute if rcs is not determine if installation is on rcs or local (or both) - this should be done for efficiency in order to uninstall background.everyMinute rcsAggregatorCacheData.prefs.path should be set by install or by communication with rcs allow user to select frequency of cache update. for now it runs at 0,15,30,and45 selectively run updateCachelist, on startup and after rssHotlist has changed, instead of every 15 minutes in everyMinute make sure xml request script runs immediately after radioCommunityServerSuite.rssHotlist.threadScript, if it goes through for now, don't schedule another call, since there could be a conflict between rebuilding the rssHotlist if it runs more often; later we should update at least every 15 minutes from client, should check rcs if it supports caching before trying turn on rssHotlist if caching is turned on (and warn user) Notes rcsAggregatorCacheSuite.modified table contains routines from Radio.root that were copied and changed for use in the tool. all modified lines will be suffixed with //MODIFIED Completed !!in rcs, cache table created manually!! !!prefs in current .root were set manually!! radio.hotlist.reload() - check to see if it's run periodically, if not call once per day or on startup or perhaps before every readAllServices hotlist info is in hotlistData.top100.[001-100]. translate this table into a list of xmlUrls first, for xml.rss.readService to check. initialization routine should add 'enabled' pref for server/client install scheduled routines - no it shouldn't - this should be done in enable in rcs.cacheRss, thread call to request and store prefs turn off/on caching; this will involve (un)installing scheduled routines (like everyMinute) as a precaution, readService scripts will double check this pref on startup, run rcs readXml in client.install, check that this is not overwritten, or undone, when aggregator is installed or inactivated rcs readXml routine should try to limit number of threads, like readAllServices check that the rcs has the rssHotlist set up clean up the prefs function! Z`梗俎公  "; G  %躬&1 9author Mikel Maron authorEmailmikel_maron@yahoo.com descriptionaWith this tool, the Radio Community Server can act as a RSS Cache for Radio Userland Aggregators.DocsV homePageUrl=http://radio.weblogs.com/0100875/outlines/rcsAggregatorCache/namercsAggregatorCache releaseDateversion0.9Arial 梗俎梗俎擫ANDwinThis script runs every hour. r Arial 梗佥恭VO atLANDwinThis script runs every minute. rcsAggregatorCacheSuite.init(rcs:true) if rcsAggregatorCacheData.prefs.rcsenabled if not tcp.isOffline() local (flscan = false) bundle //set flscan if not defined (system.temp.rcsAggregatorCache) new (tabletype, @system.temp.rcsAggregatorCache) if not defined (system.temp.rcsAggregatorCache.flStarted) flscan = true system.temp.rcsAggregatorCache.flStarted = true else local (day, month, year, hour, minute, second) date.get (clock.now (), @day, @month, @year, @hour, @minute, @second) if (minute % 15 == 0) flscan = true if flscan rcsAggregatorCacheSuite.rcs.updateCachelist() rcsAggregatorCacheSuite.rcs.cacheRss() Arial 梗俎梗俎擫ANDwinThis script runs every night. {w.梗佥工鏒  ". 9 everyHourp everyMinute everyNight{ 梗俎攻> 梗俎梗俎 梗俎恭V nj.梗俎梗俎  ' ,fileMenuradio $user PnN)Arial 躬`+公w vHLANDwinon checkRcsForCache () local(path=rcsAggregatorCacheData.prefs.path,server=user.radio.prefs.defaultCloud.server,port=user.radio.prefs.defaultCloud.port) local(result=tcp.httpClient(server:server,port:port,path:path)) local (ok = string.patternMatch("HTTP/1.1 200 OK",result) || string.patternMatch("HTTP/1.0 200 OK",result)) return ok local (params = {}); local (struct) try struct = xml.rpc(server,port,"rcsAggregatorCache.isEnabled",@params) if defined (struct.enabled) && struct.enabled return true else return false checkRcsForCache() ZArial 梗,攻+ ELANDwinon convertHotlist () Put the hotlist in a lookup table for efficient access rcsAggregatorCacheSuite.init(client:true) local adrhotlist = @hotlistData sizelist = sizeOf(adrhotlist^.top100) adrhotlookup = @rcsAggregatorCacheData.hotlist i; xmlUrl new (tabletype, adrhotlookup) for i = 1 to sizelist if defined (adrhotlist^.top100[i].xmlUrl) xmlUrl = adrhotlist^.top100[i].xmlUrl adrhotlookup^.[xmlUrl] = true convertHotlist() !0yArial 工鈀公Ws㎜ANDwinon install (uninstall=false) if not uninstall user.scheduler.everyMinute.aggregator = @rcsAggregatorCacheSuite.modified.xml.aggregator.everyMinute rcsAggregatorCacheData.prefs.clientenabled = true else user.scheduler.everyMinute.aggregator = @system.verbs.builtins.xml.aggregator.everyMinute rcsAggregatorCacheData.prefs.clientenabled = false install(uninstall:true) !xArial 梗镅公uX+ 軱ANDwinon rcsCacheRequest (url) try pulling from the rcs cache rcsAggregatorCacheSuite.init(client:true) if ! defined (rcsAggregatorCacheData.hotlist.[url]) return "" local (s) try local server = user.radio.prefs.defaultCloud.server path = rcsAggregatorCacheData.prefs.path + "?rss=" + string.urlEncode(url) port = user.radio.prefs.defaultCloud.port s = string.httpResultSplit (tcp.httpClient (server:server, path:path, port:port, timeOutTicks:60*30, flmessages:false, ctFollowRedirects:5)) bundle //comple the xml text, set the format local (xstruct, adrxstruct = @xstruct) try //compile the xmltext xml.compile (s, adrxstruct) return s else return "" 8梗镡躬f&  ,8 @L \checkRcsForCache convertHotlist#install&rcsCacheRequest(C5Arial 攻4公{["^;齃ANDwinon init (client=false,rcs=false) if not defined (rcsAggregatorCacheData.stats) new (tabletype, @rcsAggregatorCacheData.stats) if client if not defined (rcsAggregatorCacheData.hotlist) new (tabletype, @rcsAggregatorCacheData.hotlist) if not defined (rcsAggregatorCacheData.prefs.clientenabled) rcsAggregatorCacheData.prefs.clientenabled = false if not defined (rcsAggregatorCacheData.prefs.path) rcsAggregatorCacheData.prefs.path = "/rcsAggregatorCache/serveRss" if rcs if not defined (rcsAggregatorCacheData.cache) new (tabletype, @rcsAggregatorCacheData.cache) if not defined (rcsAggregatorCacheData.prefs.rcsenabled) rcsAggregatorCacheData.prefs.rcsenabled = false if not defined (rcsAggregatorCacheData.prefs.maxThreads) rcsAggregatorCacheData.prefs.maxThreads = 4 if not defined (rcsAggregatorCacheData.stats.lastRun) new (tabletype, @rcsAggregatorCacheData.stats.lastRun) if not defined (rcsAggregatorCacheData.stats.lastRun.ctThreads) rcsAggregatorCacheData.stats.lastRun.ctThreads = 0 if not defined (user.betty.rpcHandlers.rcsAggregatorCache) user.betty.rpcHandlers.rcsAggregatorCache = @rcsAggregatorCacheSuite.rcs.rpcHandlers if not defined (user.soap.rpcHandlers.rcsAggregatorCache) user.soap.rpcHandlers.rcsAggregatorCache = @rcsAggregatorCacheSuite.rcs.rpcHandlers C ]Arial 梗俎恭S楲ANDwinwebbrowser.openURL ( "http://127.0.0.1:5335/rcsAggregatorCache" ) webbrowser.bringtofront () $Arial 梗俎恭R楲ANDwinRcsAggregatorCache View in Browser 3ip4VArial .@_p"Arial 竉公_-0SeLANDwinChanges 12/4/01; 12:42:47 PM by DW If enabled, do an aggregator run at startup. 12/2/01; 1:46:56 AM by JES Respect offline status. 11/30/01; 2:13:29 PM by DW Don't run the aggregator scan if one is already running. 10/30/01; 7:01:37 AM by DW If running in Radio, send radio.log.add to xml.aggregator.readAllServices as its log callback. 10/19/01; 7:58:47 PM by DW10/19/01; 7:58:48 PM by DW Updated to call the archiver. local (flstartup = false) bundle //set flstartup if not defined (system.temp.xmlAggregator) new (tabletype, @system.temp.xmlAggregator) if not defined (system.temp.xmlAggregator.flStarted) flstartup = true system.temp.xmlAggregator.flStarted = true local (day, month, year, hour, minute, second) date.get (clock.now (), @day, @month, @year, @hour, @minute, @second) if not tcp.isOffline () local (adrdata = xml.aggregator.init ()) if adrdata^.prefs.enabled local (flscan = false) bundle //set flscan if flstartup and adrdata^.prefs.flScanOnStartup flscan = true else if minute == adrdata^.prefs.minuteToRead flscan = true if flscan local (adrcallback = nil) if system.environment.isRadio adrcallback = @radio.log.add try radio.hotlist.reload() //MODIFIED rcsAggregatorCacheSuite.client.convertHotlist() //MODIFIED xml.aggregator.readAllServices (adrlogcallback:adrcallback) //COMMENTED rcsAggregatorCacheSuite.modified.xml.aggregator.readAllServices(adrlogcallback:adrcallback) //MODIFIED xml.aggregator.updateSubscriptions () //request notification for channels that support clouds adrdata^.settings.dateLastRead = clock.now () filemenu.savemyroot (adrdata) if minute == adrdata^.prefs.minuteToArchive if adrdata^.prefs.flArchive //archiving is enabled xml.aggregator.updateArchives () everyMinute() ZprArial *竉梗) l6@酟ANDwinon readAllServices (maxStories=infinity, adrlogcallback=nil, adrdata=nil) Changes: 12/27/01; 1:31:55 PM by PBS Pass adrdata to xml.aggregator.unsubscribeErrantServices, xml.aggregator.clearCache. When calling preScan and postScan callbacks, call scripts in adrdata^.callbacks, not in aggregatorData.callbacks. 12/4/01; 7:22:27 PM by DW Call scripts in aggregatorData.callbacks.preScan before doing scan, in aggregatorData.callbacks.postScan after finishing scan. 12/3/01; 6:06:45 PM by DW Call xml.aggregator.unsubscribeErrantServices before scanning. 11/30/01; 2:18:19 PM by DW Set a boolean, adrdata^.stats.lastRun.flScanRunningNow, when you start and finish. 11/7/01; 7:04:22 PM by DW Change system.temp.aggregator to adrdata^.stats.lastRun. 10/30/01; 7:35:53 AM by DW Call radio.aggregator.preCacheNewsPage after scan if running in Radio. 10/26/01; 7:16:09 PM by DW Change final status message to include the number of stories. 8/14/01; 7:32:07 PM by JES New optional parameter, adrdata, is the address for aggregator data storage. Default is to get the address by calling xml.aggregator.init. 2/23/01; 2:11:17 PM by PBS New optional parameter, maxStories. It's not a hard maximum, the number of new stories may go a little over. We don't stop compiling all the stories in a service just because we're going a little over the limit. This is used at first time root run, so we don't read 250 stories. 1/6/01; 8:43:58 PM by DW Logging code is now controlled by a preference. 1/2/01; 5:02:11 PM by DW Save services list after scan is complete. 12/30/00; 7:58:28 PM by DW Added logging code. Sunday, December 17, 2000 at 10:27:43 AM by DW Created local (startticks = clock.ticks ()) if adrdata == nil adrdata = xml.aggregator.init () else xml.aggregator.init () local (adrservices = @adrdata^.services) xml.aggregator.unsubscribeErrantServices (adrdata) //unsubscribe from channels with too many errors //PBS 12/27/01: pass adrdata on statusmsg (s) adrdata^.stats.lastRun.status = s msg (clock.now () + "; " + s) bundle //init stats adrdata^.stats.lastRun.ctThreads = 0 adrdata^.stats.lastRun.lastChannelRead = "" adrdata^.stats.lastRun.ctNewStories = 0 adrdata^.stats.lastRun.flScanRunningNow = true on doCallbacks (adrtable) local (adrscript) for adrscript in adrtable try while typeof (adrscript^) == addresstype adrscript = adrscript^ adrscript^ () doCallbacks (@adrdata^.callbacks.preScan) local (randomtable) //read the channels in random order bundle //initialize random table local (adritem) new (tabletype, @randomtable) for adritem in adrservices loop //add service to random table adrrandom = @randomtable.[string (random (0, 32767))] if not defined (adrrandom^) adrrandom^ = nameof (adritem^) break local (cttotalthreads = 0) bundle //main loop local (adrrandom, adrservice) for adrrandom in @randomtable local (url = adrrandom^) adrservice = @adrservices^.[url] while adrdata^.stats.lastRun.ctThreads >= adrdata^.prefs.maxThreads //wait for threadcount to drop below the max thread.sleepFor (1) adrdata^.stats.lastRun.ctThreads++ thread.callScript ("xml.aggregator.readService", {url, adrservices}) //COMMENTED thread.callScript ("rcsAggregatorCacheSuite.modified.xml.aggregator.readService", {url, adrservices}) //MODIFIED cttotalthreads++ statusmsg ("Scanning. " + adrdata^.stats.lastRun.ctThreads + " current threads; " + cttotalthreads + " total threads; " + adrdata^.stats.lastRun.ctNewStories + " new stories.") if adrdata^.stats.lastRun.ctNewStories >= maxStories break while adrdata^.stats.lastRun.ctThreads > 0 statusmsg ("Waiting for threads to finish: " + adrdata^.stats.lastRun.ctThreads + " threads.") thread.sleepfor (1) bundle //add an item to the daily log if adrdata^.prefs.flLogHourlyScan if adrlogcallback != nil local (htmltext = cttotalthreads + " channels read, " + adrdata^.stats.lastRun.ctNewStories + " new stories.") adrlogcallback^ ("Hourly scan", htmltext, startticks) xml.aggregator.clearCache (adrdata) //PBS 12/27/01: pass adrdata bundle //pre-cache the News page, if running in Radio if system.environment.isRadio radio.aggregator.preCacheNewsPage () filemenu.savemyroot (adrservices) statusmsg ("Last scan complete at " + date.timestring () + "; " + adrdata^.stats.lastRun.ctNewStories + " new stories.") adrdata^.stats.lastRun.whenComplete = clock.now () adrdata^.stats.lastRun.flScanRunningNow = false doCallbacks (@adrdata^.callbacks.postScan) msg ("") bundle //test code readAllServices (adrlogcallback:@radio.log.add) readAllServices () Zmk Arial 箹工贋#D頛ANDwinon readService (url, adrservices) Changes 8/31/02; 12:08:59 PM by DW Undo the referer stuff done in June. It didn't get completed, and made a mess of referer logs of sites that our users read. So, for Radio users, instead of setting the referer to the page on the Frontier site with the url of the user's site as a parameter, we point to the What is a News Aggregator page on the Radio site, with no parameters. 6/5/02; 1:19:05 PM by DW Switch back to the appSignatureUrl, if it's the default value, add a param to the URL indicating where the user's weblog is. 6/5/02; 11:11:17 AM by DW If possible, set the referer to the url of the user's weblog. If not possible, fall back to the original behavior. 11/7/01; 7:10:37 PM by DW Change system.temp.aggregator to adrdata^.stats.lastRun. local (adrdata = xml.aggregator.init ()) local (referer = adrdata^.prefs.appSignatureUrl) if system.environment.isRadio //8/31/02 by DW if referer == "http://frontier.userland.com/xmlAggregator" try {referer = referer + "?userWeblog=" + string.urlencode (radio.weblog.geturl ())} referer = "http://radio.userland.com/newsAggregator" local (adrservice = xml.rss.readService (url, adrservices, referer, @xml.aggregator.storyArrivedCallback)) //COMMENTED local (adrservice = rcsAggregatorCacheSuite.modified.xml.rss.readService (url, adrservices, referer, @xml.aggregator.storyArrivedCallback)) //MODIFIED adrdata^.stats.lastRun.ctThreads-- try {adrdata^.stats.lastRun.lastChannelRead = adrservice^.compilation.channeltitle} return (true) bundle //test code readService ("http://jrobb.userland.com/rss.xml", @scratchpad) m~.梗趥梗阽  (4 @ everyMinute5readAllServices>@ readServiceS` JArial $筂仝攻A* CLANDwinon readService (url, adrservices, referer="", adrStoryArrivedCallback=nil, flSaveData=false) Changes 3/24/02; 8:49:35 AM by DW New optional parameter, flSaveData, passed to xml.rss.compiledService. If true, each item has a data sub-table, containing the non-mashed elements from the items in the XML feed. 12/17/01; 4:47:26 PM by DW Support username-password's encoded in the URL, per request by Doug Kaye. 12/3/01; 5:43:04 PM by DW Manage ctErrors, ctConsecutiveErrors, timeLastError. Thursday, December 28, 2000 at 7:26:03 AM by DW Added adrStoryArrivedCallback, it's called when a new story has arrived. You can store it in a database, or whatever else you might want to do. Sunday, December 17, 2000 at 10:16:37 AM by DW Created. Reads a URL into a service table. Referer is passed as a header if non-empty, allows the server to track references. It shows up in the /stats/referers page on Manila sites. local (adrservice = xml.rss.initService (url, adrservices)) //make sure all fields are init'd try local (s) //MODIFIED rcsAggregatorCacheSuite.init(client:true) //MODIFIED if (rcsAggregatorCacheData.prefs.clientenabled == true) //MODIFIED try s = rcsAggregatorCacheSuite.client.rcsCacheRequest(url) //MODIFIED if (s == nil || sizeOf(s) == 0) //MODIFIED local (headers, adrheaders = nil) //MODIFIED if sizeof (referer) > 0 //MODIFIED new (tabletype, @headers) headers.referer = referer adrheaders = @headers local (urllist = string.urlSplit (url), server = urllist [2], path = urllist [3], username = "", password = "") //MODIFIED bundle //set username, password, if they were present in the URL //MODIFIED Example: http://doug:guacamole@www.infrastrat.com/ht/rss.xml if server contains "@" local (s = string.nthfield (server, "@", 1)) username = string.nthfield (s, ":", 1) password = string.nthfield (s, ":", 2) server = string.nthfield (server, "@", 2) s = string.httpResultSplit (tcp.httpClient (server:server, path:path, timeOutTicks:60*30, flmessages:false, ctFollowRedirects:5, adrHdrTable:adrheaders, username:username, password:password)) //MODIFIED local (now = clock.now ()) adrservice^.timeLastRead = now adrservice^.ctReads++ local (flchanged = true) if defined (adrservice^.xmltext) if s == adrservice^.xmltext flchanged = false if flchanged adrservice^.ctChanges++ adrservice^.xmltext = s adrservice^.timeLastChange = now local (errorstring = "") semaphore.lock (this, 3600) //keep stories from each channel in a group try xml.rss.compileService (adrservice, flSaveData, adrStoryArrivedCallback) else errorstring = tryerror semaphore.unlock (this) if errorstring != "" scripterror (errorstring) try {delete (@adrservice^.error)} adrservice^.ctConsecutiveErrors = 0 else adrservice^.error = tryError adrservice^.errorbak = tryError //keep error around to track Groups bug adrservice^.ctErrors++ adrservice^.ctConsecutiveErrors++ adrservice^.timeLastError = clock.now () scratchpad.service = adrservice^ return (adrservice) bundle //test code readService ("http://www.infrastrat.com/ht/rss.xml", @scratchpad) //should get an error readService ("http://doug:guacamole@www.infrastrat.com/ht/rss.xml", @scratchpad) readService ("http://radio.weblogs.com/0001015/categories/radio7.1/rss.xml", @scratchpad) `:6梗邛梗 readService[:SO$梗趘梗   aggregator[rsskS2.梗梗邬 xmlk_2+ Arial 攻D公^S!LANDwinon prefs () rcsAggregatorCacheSuite.init(client:true,rcs:true) local (pta = html.getpagetableaddress ()) pta^.title = "RCS Aggregator Cache" pta^.template = rcsAggregatorCacheWebsite.["#templates"].default local (args); new (tabletype, @args) case pta^.method "POST" webserver.parseargs (pta^.requestBody, @args) "GET" webserver.parseargs (pta^.searchargs, @args) local(serversupported,serversupportedmsg,clientsupported,clientsupportedmsg) bundle //check server supported if (rcsAggregatorCacheSuite.rcs.checkRcsInstalled()) if (rcsAggregatorCacheSuite.rcs.checkRcsRSSHotlist()) serversupported = true serversupportedmsg = "RCS present, and RSS Hotlist enabled" else serversupported = false serversupportedmsg = "RSS Hotlist is not enabled. You can enable it here" else serversupported = false serversupportedmsg = "RCS is not installed locally. Install from http://rcs.userland.com/" if (serversupported == false) && rcsAggregatorCacheData.prefs.rcsenabled rcsAggregatorCacheSuite.rcs.install(uninstall:true) bundle //check client supported if not tcp.isOffline() if (rcsAggregatorCacheSuite.client.checkRcsForCache()) clientsupported = true clientsupportedmsg = "Your RCS does support RSS caching" else clientsupported = false clientsupportedmsg = "Your RCS does not presently support RSS caching. Please notify the RCS administrator of your interest in RSS caching." if (rcsAggregatorCacheData.prefs.clientenabled) rcsAggregatorCacheSuite.client.install(uninstall:true) else clientsupported = true clientsupportedmsg = "Could not determine if caching is supported on your RCS" if defined (args.submit) if defined (args.clientenabled) if clientsupported rcsAggregatorCacheSuite.client.install() else rcsAggregatorCacheSuite.client.install(uninstall:true) else if (rcsAggregatorCacheData.prefs.clientenabled) rcsAggregatorCacheSuite.client.install(uninstall:true) if defined (args.rcsenabled) if serversupported rcsAggregatorCacheSuite.rcs.install() else rcsAggregatorCacheSuite.rcs.install(uninstall:true) else if (rcsAggregatorCacheData.prefs.rcsenabled) rcsAggregatorCacheSuite.rcs.install(uninstall:true) local (htmltext="") on add (s) htmltext = htmltext + s add add("
") add("") add("") add("") add("
") add(" Enable Client Caching.
    " + clientsupportedmsg + "

") add(" Enable RCS Caching.
 " + serversupportedmsg + "
") add ("
") return htmltext +Arial 工功3*FuLANDwinon cacheRss () rcsAggregatorCacheSuite.init(rcs:true) local adrcache = @rcsAggregatorCacheData.cache i sizecache = sizeOf (adrcache^) adrdata = @rcsAggregatorCacheData cttotalthreads = 0 xmlUrl on statusmsg (s) adrdata^.stats.lastRun.status = s msg (clock.now () + "; " + s) for i = 1 to sizecache try while adrdata^.stats.lastRun.ctThreads >= adrdata^.prefs.maxThreads //wait for threadcount to drop below the max thread.sleepFor (1) adrdata^.stats.lastRun.ctThreads++ xmlUrl = nameOf(adrcache^[i]) thread.callScript(@rcsAggregatorCacheSuite.rcs.retrieveXml,{xmlUrl,adrcache}) cttotalthreads++ statusmsg ("Scanning. " + adrdata^.stats.lastRun.ctThreads + " current threads; " + cttotalthreads + " total threads") while adrdata^.stats.lastRun.ctThreads > 0 statusmsg ("Waiting for threads to finish: " + adrdata^.stats.lastRun.ctThreads + " threads.") thread.sleepfor (1) cacheRss()  TArial 躬ij躬j@LANDwinon checkRcsInstalled () return defined (user.tools.databases.radioCommunityServer) >Arial 躬j4躬ju9狶ANDwinon checkRcsRSSHotlist () if defined(radioCommunityServerData.rssHotlist.prefs.enabled) return radioCommunityServerData.rssHotlist.prefs.enabled else return false >0Arial 攻婀玕k B)pLANDwinon install (uninstall=false) if (uninstall) rcsAggregatorCacheData.prefs.rcsenabled = false else if defined (system.temp.rcsAggregatorCache) delete (@system.temp.rcsAggregatorCache) rcsAggregatorCacheData.prefs.rcsenabled = true install() Arial 工?功2_"覮ANDwinon retrieveXml (xmlUrl,adrcache) rcsAggregatorCacheSuite.init(rcs:true) try local (urllist = string.urlSplit (xmlUrl), server = urllist [2], path = urllist [3], username = "", password = "") bundle //set username, password, if they were present in the URL Example: http://doug:guacamole@www.infrastrat.com/ht/rss.xml if server contains "@" local (s = string.nthfield (server, "@", 1)) username = string.nthfield (s, ":", 1) password = string.nthfield (s, ":", 2) server = string.nthfield (server, "@", 2) local (s = string.httpResultSplit (tcp.httpClient (server:server, path:path, timeOutTicks:60*30, flmessages:false, ctFollowRedirects:5, username:username, password:password))) local (now = clock.now()) adrcache^.[xmlUrl].timeLastRead = clock.now() adrcache^.[xmlUrl].ctReads++ local (flchanged = true) if defined (adrcache^.[xmlUrl].xmltext) if (s != adrcache^.[xmlUrl].xmltext) flchanged = false if flchanged adrcache^.[xmlUrl].ctChanges++ adrcache^.[xmlUrl].xmltext = s adrcache^.[xmlUrl].timeLastChange = now if rcsAggregatorCacheData.stats.lastRun.ctThreads > 0 rcsAggregatorCacheData.stats.lastRun.ctThreads-- P*Arial 躬v肮獁@Z覮ANDwinon isEnabled () local (struct); new (tabletype, @struct) if rcsAggregatorCacheData.prefs.rcsenabled struct.enabled = true else struct.enabled = false return (struct) P84躬v柟獀 isEnabled垊8~Arial 工 公l塋ANDwinon serveRss () rcsAggregatorCacheSuite.init() if (rcsAggregatorCacheData.prefs.rcsenabled == false) scriptError("rcs aggregator cache not enabled") local (pta = html.getpagetableaddress ()) pta^.template = rcsAggregatorCacheWebsite.["#templates"].rsstemplate pta^.pageHeader = "" local (args); new (tabletype, @args) local (rss) case pta^.method "POST" webserver.parseargs (pta^.requestBody, @args) "GET" webserver.parseargs (pta^.searchargs, @args) if defined (args.rss) rss = args.rss local (xmlUrl = string.urlDecode(rss)) if defined (rcsAggregatorCacheData.cache.[rss].xmltext) return rcsAggregatorCacheData.cache.[rss].xmltext else return "" JArial 工羘攻$LANDwinon updateCachelist (expireminutes=120) rcsAggregatorCacheSuite.init(rcs:true) local adrcache = @rcsAggregatorCacheData.cache adrhotlist = @radioCommunityServerData.rssHotlist.sources sizehotlist = sizeOf(adrhotlist^) i xmlUrl add rssHotlist to cache list for i = 1 to sizehotlist xmlUrl = nameOf(adrhotlist^[i]) if not defined (adrcache^.[xmlUrl]) new (tabletype, @adrcache^.[xmlUrl]) adrcache^.[xmlUrl].lastAdded = clock.now() adrcache^.[xmlUrl].xmlUrl = xmlUrl if not defined (adrcache^.[xmlUrl].ctReads) adrcache^.[xmlUrl].ctReads = 0 if not defined (adrcache^.[xmlUrl].ctChanges) adrcache^.[xmlUrl].ctChanges = 0 clear out entries older than expireminutes local sizecache lastAdded cachecopy = adrcache^ copy cache table so we can delete from real one. there must be a more elegant way... sizecache = sizeOf(cachecopy) for i =1 to sizecache lastAdded = cachecopy[i].lastAdded xmlUrl = nameOf(cachecopy[i]) if (number (date (clock.now())) - number (date (lastAdded)) > expireminutes) delete (@adrcache^[i]) bundle //test updateCachelist() /+`工罰躬v  '3 FR Zf r~ cacheRssz3checkRcsInstalled~checkRcsRSSHotlistinstall retrieveXml偼 rpcHandlers夃serveRss$updateCachelist嵭/<Arial 梗俎躬w; a芁ANDwinThis script returns a string which appears in the Status Center on the Radio UserLand home page. rcsAggregatorCacheSuite.init(rcs:true,client:true) local (message = "") if (rcsAggregatorCacheData.prefs.clientenabled) message = "
RSS Caching enabled for Client" if (rcsAggregatorCacheData.prefs.rcsenabled) message = message + "
RSS Caching enabled for this RCS" if (string.length(message) == 0) message = "
RSS Caching is not enabled" return message +'j梗佥躬v  !- 4@ EQ Vb kw } backgroundq callbacks |client,finit-menu5modifiedkprefskrcs揹statusCenterMessage敓+Arial 梗俎梗俎圠ANDwinThis script runs in its own thread. loop //loop until the application quits Add your code here. thread.sleepFor (10) //sleep for ten seconds TP.梗俎梗俎 ctenabledscript榚TtArial 禙F2甃ANDwinon finalFilter (pta) pta^.renderedText = string.replace (pta^.renderedText, "[[title]]", "") return (true) ����������������������������� ���-���� MS Sans Serif��������������������� ��������������Eu碻轖�������2LAND�������win������������on firstFilter (adrPageTable) return (true) �����������������������*��[���� MS Sans Serif��������������������� ��������������Eu迭7a���m���L�3*lLAND�������win������������on pageFilter (pta) if typeOf (pta^.template) == outlineType or typeOf (pta^.template) == wpTextType table.assign (@pta^.template, string (pta^.template)) if typeOf (pta^.template) == stringType and not pta^.indirectTemplate pta^.template = string.replace (pta^.template, "<title>", "[[title]]") html.addPageToGlossary (pta) return (true) �����������������������������������������������y���.���E级F�������� ���� ��� ����$���0 ����; finalFilter�������欚 firstFilter�������汑 pagefilter�������浄��V MS Sans Serif��������������������� �����������P�E��$����������������������������N�������J������迭7üＹ�������� ����url���$http://127.0.0.1/rcsAggregatorCache/���N�������������$���工杖工杖�������� ���� ���} ����linetext���p["C:\\Program Files\\Radio Community Server\\Tools\\rcsAggregatorCache.root"].rcsAggregatorCacheWebsite.serveRsspath��� serveRss.wsf����������������$���恭W�������� ���� ���r ����wlinetext���e["C:\\Program Files\\Radio UserLand\\Tools\\rcsAggregatorCache.root"].rcsAggregatorCacheWebsite.indexpath��� index.wsf����������������$���工工�������� ���� ���u ����zlinetext���h["C:\\Program Files\\Radio UserLand\\Tools\\rcsAggregatorCache.root"].rcsAggregatorCacheWebsite.serveRsspath��� serveRss.wsf������O�������K���$���工谠工谠�������� ���� ��� ����linetext���Homepath��� index.wsf���O���_�������[���$���恭_抗�������� ���� ���! ����&linetext���RCS Aggregator Cachepath��� index.wsf���_��6���������B�������恭_�������� ����q���} ������� ���X��d ���i��u ���妏["C:\\Program Files\\Radio Community Server\\Tools\\rcsAggregatorCache.root"].rcsAggregatorCacheWebsite.serveRss�������烓e["C:\\Program Files\\Radio UserLand\\Tools\\rcsAggregatorCache.root"].rcsAggregatorCacheWebsite.index�������熋h["C:\\Program Files\\Radio UserLand\\Tools\\rcsAggregatorCache.root"].rcsAggregatorCacheWebsite.serveRss�������Home�������RCS Aggregator Cache���������VArial����������������������������� �����������������������������������������������6����������W����������灃������������space���/GIFfGIF89a�������!����,�������D�;��VArial����������������������������� ������������������������������������������������������������V�������瀱�������� ������� �������"�������+ ����0���: ����C���L ����Q���[ ����aalink���008000bgcolor���FFFFFFflRenderlink���0000FFspaceGif���spacetext���000000vlink���800080��VArial����������������������������� ���������������������������������������������������������~��� ���Arial����������������������������� ��������������禙v���8����M LAND�������win������������<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <title >{title} <% metaTags() %>
<%radioBadge%> Community Server :: Aggregator Cache <% radio.userInterface.helpIcon() %>
 



<%bodytext%>

<%radio.macros.editorsOnlyMenu ("black")%>
Arial 禙玤頛ANDwin<% bodytext %> XT$恭_2公g  default1 rsstemplateXn攻VArial n'Arial 攻恭^圠ANDwinreturn rcsAggregatorCacheSuite.prefs()  >Arial 工袹公ib/螸ANDwinon serveRss () return rcsAggregatorCacheSuite.rcs.serveRss() &j迭7惞玤  * 4@ HT [g r~ #filters澙#ftpSite灍 #glossary#images#prefs #templates玬#toolsindex琄serveRssVVerdana ??8UB梗佥公{f # :F ^j rcsAggregatorCacheDatarcsAggregatorCacheInforcsAggregatorCacheSuite.rcsAggregatorCacheThread檺rcsAggregatorCacheWebsiteVArial ]'mU疺Arial MS Sans Serif + Times New Roman) 3Arial B~Arial MS Sans Serif