/[meteor]/trunk/meteord
This is repository of my old source code which isn't updated any more. Go to git.rot13.org for current projects!
ViewVC logotype

Annotation of /trunk/meteord

Parent Directory Parent Directory | Revision Log Revision Log


Revision 62 - (hide annotations)
Thu Nov 27 00:33:21 2008 UTC (15 years, 4 months ago) by andrew.betts
Original Path: googlecode.com/svn/trunk/meteord
File size: 9135 byte(s)
1 Fixed: Added SIGPIPE handler.  We noticed that under heavy load
Meteor receives SIGPIPEs from the OS, suspected to relate to clients
that have just disconnected the moment Meteor attempts to write to the
socket.  This caused Meteor to crash.
2 Fixed: Long polling multiple channels no longer causes the loop to
die and restart when some channels have messages queued for delivery.
3 Fixed: Over time, Meteor 'collected' connections from clients that
never got disconnected even if MaxTime was set.  This happened if the
client concerned sent a header with no terminating blank line.  Meteor
kept waiting for the rest of the header, which never arrived, and
therefore the client remained in limbo, never subjected to the MaxTime
time limit because it had not yet become a subscriber.  Clients are
now allowed 30 seconds to send a valid request header.
4 Fixed: If only one message existed on the server, the JS client
would continue to request it again and again, because it has message
ID 0, and the JS client considered this an invalid message ID.
5 Fixed: Corrected some comments in file headers

6 Changed: MaxMessages has been renamed to CloseOnEvent and functions
in a similar, but not quite identical way.  Thanks to Matthew Haak,
who pointed out the extreme confusingness of MaxMessages and a bug
that has resulted in Fix 2 above.  Setting CloseOnEvent to any value
that evaluates to true will cause Meteor to close subscriber
connections after at least one message has been sent and there are no
further messages pending.  This is identical to MaxMessages for values
of 0 and 1, but where MaxMessages is set to a value higher than one,
replacing it with CloseOnEvent with the same value will act as though
it were set to one.  The intent of MaxMessages was to enable long-
polling (and it is used by the JS client in that way), and
CloseonEvent is a drop in replacement for that behaviour.
7 Changed: Meteor JS client now uses dynamic <SCRIPT> tags for all
polling behaviours, rather than XHR.  This enables it to make poll
requests cross-domain (see 13)
8 Changed: Meteor JS client now abstracts timestamp lookups to a
dedicated method.
9 Changed: Default HeaderTemplates no longer include cache busting
headers, since all meteor requests contain a millisecond timestamp and
so no client makes the same request twice.  These were therefore
simply chewing up bandwidth.
10 Changed: Date strings used for logging debug messages are cached to
avoid numerous expensive lookups to localtime().
11 Changed: Channel info is only sent in a response if the client does
not request a restart from a specified ID.  The logic being that if
the client knows the ID they want to start from, they have already
made previous requests and have the channel information they need.
Bandwidth saving measure.

12 Added: JS client now has a Meteor.isSupportedBrowser() method,
which you can call to detemine whether Meteor will run in the user's
browser version.
13 Added: JS client can now use different hosts for polling and
streaming.  This is only really useful if your website is on a domain
that has a lot of cookies, and you don't want to send them in every
poll request.  Removing cookies from request headers can reduce the
size of the request significantly.  We find that with cookies included
Meteor poll requests are usually larger than the responses.  To use,
set Meteor.pollhost.  Meteor.pollhost can be any domain, while
Meteor.host must be a subdomain of your website hostname.
14 Added: Config file now supports new 'FooterTemplate' parameter, for
a string to send just before the connection to the subscriber is
closed.  This is in support of change 7.
15 Added: Better inline documentation for ChannelInfoTemplate config
parameter
16 Added: Log output includes connection IDs corresponding to the file
inode for each connection
17 Added: New controller command LISTCONNECTIONS, produces a newline
delimited list of all currently connected clients, and for each one
displaying "ConnectionID IPAddress ClientType [SubscriberID]"
18 Added: New controller command DESCRIBE, takes a ConnectionID as a
parameter, and outputs numerous statistics about that particular
client, including number of messages sent/received, user agent, IP
address, time connected, time remaining until MaxTime etc.
19 Added: New controller comment LISTSUBSCRIBERS, produces a newline
delimited list of all currently connected streaming subscribers, and
for each one displaying "SubscriberID IPAddress Starttime TimeLimit
TimeRemaining MessageCount UserAgent"
20 Added: SHOWSTATS command produces the following additional stats:
connection_count: total current connections, real_subscribers: total
of number of currently connected streaming subscribers plus the number
of unique polling connections seen in the last 60 seconds.
21 Added: STDERR outputs prior to every exit() for debugging purposes
22 Added: The UDP server is now considered stable, and is the best way
of broadcasting messages to lots of Meteor nodes simultaneously and
efficiently. 


1 knops.gerd 11 #!/usr/bin/perl -w
2     ###############################################################################
3     # Meteor
4     # An HTTP server for the 2.0 web
5     # Copyright (c) 2006 contributing authors
6     #
7     # The Meteor daemon
8     #
9     # Main program should call Meteor::Config::setCommandLineParameters(@ARGV),.
10     # Afterwards anybody can access $::CONF{<parameterName>}, where
11     # <parameterName> is any valid parameter (except 'Help') listed in the
12     # @DEFAULTS array below.
13     #
14     ###############################################################################
15     #
16     # This program is free software; you can redistribute it and/or modify it
17     # under the terms of the GNU General Public License as published by the Free
18     # Software Foundation; either version 2 of the License, or (at your option)
19     # any later version.
20     #
21     # This program is distributed in the hope that it will be useful, but WITHOUT
22     # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
23     # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
24     # more details.
25     #
26     # You should have received a copy of the GNU General Public License along
27     # with this program; if not, write to the Free Software Foundation, Inc.,
28     # 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
29     #
30     # For more information visit www.meteorserver.org
31     #
32     ###############################################################################
33    
34 knops.gerd 42 ###############################################################################
35     # meterod version
36     ################################################################################
37    
38 andrew.betts 62 $::VERSION='1.06.02';
39 andrew.betts 59 $::RELEASE_DATE='not yet released';
40 knops.gerd 11
41     ###############################################################################
42     # Configuration
43     ###############################################################################
44    
45     use strict;
46    
47 knops.gerd 48 use Socket;
48    
49 knops.gerd 11 use Meteor::Syslog;
50    
51     use Meteor::Socket;
52     use Meteor::Connection;
53     use Meteor::Controller;
54     use Meteor::Subscriber;
55     use Meteor::Channel;
56     use Meteor::Document;
57     use Meteor::Config;
58    
59     $::CRLF="\r\n"; # Line separator to be used throughout all modules
60    
61     our $CONTROL_QUEUE_SIZE=5;
62     our $SUBSCRIBER_QUEUE_SIZE=20;
63    
64     our $MAIN_LOOP_TIMEOUT=60;
65     our $AGE_CHECK_INTERVALL=60;
66    
67     our $MAX_EXIT_DELAY=120;
68 knops.gerd 48
69     our $UDP_MAX_MESSAGE_SIZE=8192;
70 knops.gerd 11
71     ###############################################################################
72     # Main
73     ###############################################################################
74    
75     #
76 knops.gerd 25 # Record startup time
77     #
78     $::STARTUP_TIME=time;
79     $::STARTUP_TIME+=0; # avoid warning
80    
81     #
82 knops.gerd 11 # Program name
83     #
84     $::PGM=$0;
85     $::PGM=~s/^.*\///;
86    
87     #
88     # Handle command line options and config file
89     #
90     Meteor::Config->setCommandLineParameters(@ARGV);
91    
92     #
93     # Do something about warn and die
94     #
95     unless($::CONF{'Debug'})
96     {
97     $SIG{'__WARN__'}=\&Meteor::Syslog::myWarn;
98     $SIG{'__DIE__'}=\&Meteor::Syslog::myDie;
99     }
100    
101     &::syslog('info',"$::PGM launched!");
102    
103     #
104     # Daemonize
105     #
106     {
107     $0="$::PGM daemon";
108    
109 knops.gerd 47 my $facility=$::CONF{'SyslogFacility'} || $Meteor::Syslog::DEFAULT_FACILITY;
110    
111     unless($::CONF{'Debug'} || $facility eq 'none')
112 knops.gerd 11 {
113     # close standard file descriptors
114     close(STDIN);
115     close(STDOUT);
116     close(STDERR);
117     chdir("/");
118     umask(0);
119 andrew.betts 62
120 knops.gerd 11 # fork and exit parent
121 andrew.betts 62 print STDERR "Exit: Fork\n";
122 knops.gerd 11 exit if fork;
123     setpgrp(0, $$) if defined $SIG{TTOU};
124     $SIG{TTOU}='ignore' if defined $SIG{TTOU};
125    
126     # Avoid 'stdin reopened for output' warning with newer perls
127     open(NULL,'/dev/null');
128     <NULL> if(0);
129     open(OUT,">/var/run/$::PGM.pid");
130     print OUT "$$\n";
131     close(OUT);
132 andrew.betts 62
133 knops.gerd 11 }
134     else
135     {
136 knops.gerd 47 &::syslog('info',"PID\t%s",$$);
137 knops.gerd 11 }
138     }
139    
140     #
141     # Signal handlers
142     #
143     $::HUP=$::TERM=$::USR1=$::USR2=0;
144     $SIG{'HUP'}=sub{$::HUP=1};
145     $SIG{'TERM'}=sub{$::TERM=1};
146     $SIG{'USR1'}=sub{$::USR1=1};
147     $SIG{'USR2'}=sub{$::USR2=1};
148 andrew.betts 62 $SIG{'PIPE'}=sub{&::syslog('info',"Signal PIPE received and ignored\n");};
149    
150 knops.gerd 11 #
151     # Run server
152     #
153     my $con_counter=0;
154     my $con;
155    
156     my $controlServer=Meteor::Socket->newServer(
157     $::CONF{'ControllerPort'},
158     $CONTROL_QUEUE_SIZE,
159     $::CONF{'ControllerIP'}
160     );
161     my $controlServerFN=$controlServer->fileno();
162    
163     my $subscriberServer=Meteor::Socket->newServer(
164     $::CONF{'SubscriberPort'},
165     $SUBSCRIBER_QUEUE_SIZE,
166     $::CONF{'SubscriberIP'}
167     );
168     my $subscriberServerFN=$subscriberServer->fileno();
169    
170 knops.gerd 48 my $udpServer=undef;
171     my $udpPort=$::CONF{'UDPPort'};
172     my $udpServerFN=undef;
173     if($udpPort && $udpPort>0)
174     {
175     $udpServer=Meteor::Socket->newUDPServer(
176     $udpPort,
177     $::CONF{'UDPIP'}
178     );
179     $udpServerFN=$udpServer->fileno();
180     }
181    
182 knops.gerd 11 my $serverVector='';
183     vec($serverVector,$controlServerFN,1)=1;
184     vec($serverVector,$subscriberServerFN,1)=1;
185 knops.gerd 48 vec($serverVector,$udpServerFN,1)=1 if(defined($udpServerFN));
186 knops.gerd 11
187     my $lastAgeCheck=time;
188    
189     my $nextPing=undef;
190     if(exists($::CONF{'PingInterval'}) && $::CONF{'PingInterval'}>2)
191     {
192     $nextPing=$::CONF{'PingInterval'}+$lastAgeCheck;
193     }
194    
195     while(!$::TERM)
196     {
197     eval
198     {
199     while(!$::TERM)
200     {
201 andrew.betts 62
202    
203 knops.gerd 11 my $rVec=$serverVector;
204     my $wVec='';
205     my $eVec='';
206 knops.gerd 47
207 knops.gerd 11 my $rout;
208     my $wout;
209     my $eout;
210 knops.gerd 47
211 knops.gerd 11 Meteor::Connection->addAllHandleBits(\$rVec,\$wVec,\$eVec);
212    
213     my $timeout=$MAIN_LOOP_TIMEOUT;
214     if(defined($nextPing))
215     {
216     $timeout=$nextPing-time;
217     }
218    
219     my $result=0;
220     if($timeout>0)
221     {
222     $result=&Meteor::Socket::sselect($rout=$rVec,$wout=$wVec,$eout=$eVec,$timeout);
223     }
224    
225     if($result>0)
226     {
227     if(vec($rout,$controlServerFN,1))
228     {
229     Meteor::Controller->newFromServer($controlServer);
230     }
231     if(vec($rout,$subscriberServerFN,1))
232     {
233     Meteor::Subscriber->newFromServer($subscriberServer);
234     }
235 knops.gerd 48 if(defined($udpServerFN) && vec($rout,$udpServerFN,1))
236     {
237     &handleUPD($udpServer);
238     }
239 knops.gerd 11
240     Meteor::Connection->checkAllHandleBits($rout,$wout,$eout);
241     }
242     elsif($result<0)
243     {
244     &::syslog('crit',"Select failed: $!");
245     sleep(30);
246     }
247    
248     if($::HUP)
249     {
250     $::HUP=0;
251    
252     &::syslog('info',"Received SIGHUP, re-reading config and clearing document cache!");
253    
254     Meteor::Config->readConfig();
255     Meteor::Config->updateConfig();
256    
257     Meteor::Document->clearDocuments()
258     }
259    
260     if($::USR1)
261     {
262     $::USR1=0;
263    
264     &::syslog('info',"Received SIGUSR1, clearing channel buffers!");
265    
266     Meteor::Channel->clearAllBuffers();
267     }
268 andrew.betts 62
269 knops.gerd 11 if($::USR2)
270     {
271     $::USR2=0;
272    
273     &::syslog('info',"Received SIGUSR2, clearing document cache!");
274    
275     Meteor::Document->clearDocuments()
276     }
277 andrew.betts 62
278 knops.gerd 11 my $t=time;
279     if($t>$lastAgeCheck+$AGE_CHECK_INTERVALL)
280     {
281     my $minTimeStap=time-$::CONF{'MaxMessageAge'};
282     Meteor::Channel->trimMessageStoresByTimestamp($minTimeStap);
283     $lastAgeCheck=time;
284     $t=$lastAgeCheck;
285    
286     Meteor::Subscriber->checkPersistentConnectionsForMaxTime();
287 andrew.betts 62 Meteor::Connection->destroyBadRequests();
288 knops.gerd 11 }
289    
290     if(defined($nextPing) && $nextPing<=$t)
291     {
292     $nextPing=undef;
293    
294     Meteor::Subscriber->pingPersistentConnections();
295    
296     if(exists($::CONF{'MaxMessageAge'}) && $::CONF{'MaxMessageAge'}>2)
297     {
298     $nextPing=$::CONF{'PingInterval'}+time;
299     }
300     }
301     }
302     };
303     unless($::TERM)
304     {
305     &::syslog('alert',"$::PGM loop died (will restart in 2 seconds): $@");
306     sleep(2);
307     }
308     }
309    
310     #
311     # Proper shutdown
312     #
313     if($::TERM)
314     {
315     &::syslog('info',"Received SIGTERM, begin shutdown!");
316    
317     $subscriberServer->close();
318     $controlServer->close();
319    
320     unlink("/var/run/$::PGM.pid") unless($::CONF{'Debug'});
321    
322     Meteor::Connection->closeAllConnections();
323    
324     my $timoutAt=time+$MAX_EXIT_DELAY;
325    
326     while(Meteor::Connection->connectionCount() && time<$timoutAt)
327     {
328     my $rVec='';
329     my $wVec='';
330     my $eVec='';
331    
332     my $rout;
333     my $wout;
334     my $eout;
335    
336     Meteor::Connection->addAllHandleBits(\$rVec,\$wVec,\$eVec);
337    
338     my $result=&Meteor::Socket::sselect($rout=$rVec,$wout=$wVec,$eout=$eVec,$timoutAt-time);
339    
340     if($result>0)
341     {
342     Meteor::Connection->checkAllHandleBits($rout,$wout,$eout);
343     }
344     }
345    
346     if(my $cnt=Meteor::Connection->connectionCount())
347     {
348     &::syslog('info',"$cnt client(s) unresponsive, will shutdown anyway");
349    
350 andrew.betts 62 print STDERR "Exit: TERM Shutdown (unresponsive clients)\n";
351 knops.gerd 11 exit(1);
352     }
353    
354     &::syslog('info',"shutdown succeeded");
355    
356 andrew.betts 62 print STDERR "Exit: TERM Shutdown (clean)\n";
357 knops.gerd 11 exit(0);
358     }
359    
360     &::syslog('emerg',"$::PGM loop exited");
361    
362 knops.gerd 48 ###############################################################################
363     # Subroutines
364     ###############################################################################
365     sub handleUPD {
366     $udpServer=shift;
367    
368     my $line;
369     my $hispaddr=recv($udpServer->{'handle'},$line,$::UDP_MAX_MESSAGE_SIZE,0);
370    
371     &::syslog('debug',"udp message received: %s",$line);
372    
373     return unless($line=~s/^(\S+)\s//);
374    
375     my $cmd=$1;
376    
377     if($cmd eq 'ADDMESSAGE')
378     {
379     return unless($line=~s/^(\S+)\s//);
380    
381     my $channelName=$1;
382     my $channel=Meteor::Channel->channelWithName($channelName);
383     my $msg=$channel->addMessage($line);
384     my $msgID=$msg->id();
385     &::syslog('debug',"udp: new message added, ID %s",$msgID);
386     }
387     }
388    
389 knops.gerd 11 1;
390 andrew.betts 62 ############################################################################EOF

  ViewVC Help
Powered by ViewVC 1.1.26