tag:blogger.com,1999:blog-7449668645955818675.comments2022-04-07T01:44:11.758+02:00Patrick Allaert's Blog about Free SoftwarePatrick Allaerthttp://www.blogger.com/profile/08540817341697816823noreply@blogger.comBlogger64125tag:blogger.com,1999:blog-7449668645955818675.post-52475564755669621752016-06-28T13:53:19.471+02:002016-06-28T13:53:19.471+02:00i searched for exactly this stuff :-)
thx for post...i searched for exactly this stuff :-)<br />thx for posting<br />dont know perl, but know how to use it, it works greatDirkhttps://www.blogger.com/profile/04051066937864324360noreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-87030051632856477612015-10-12T05:31:42.291+02:002015-10-12T05:31:42.291+02:00I would like to know if there was ever a conclusio...I would like to know if there was ever a conclusion drawn as to this methods performance. I wrote a class similar to the ApcClassLoader that can be used as a decorator to implement this technique. It's in a gist <a href="https://gist.github.com/superdav42/55f53b30ca5b5c724c6b" rel="nofollow">Symlink Loader Cache</a> This will make testing and using this technique very easy for everyone, Probably a class like this would be the ideal implementation instead of modifying composer to create the symlinks.<br /><br />Please leave comments there for any improvements. Davehttps://www.blogger.com/profile/07712070839374658252noreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-11221901660862094822014-03-04T20:47:52.601+01:002014-03-04T20:47:52.601+01:00This was seriously helpful and the fact that it wo...This was seriously helpful and the fact that it works with PDO prepared statements is killer. Thanks!Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-72256064302588180682013-10-29T13:07:23.722+01:002013-10-29T13:07:23.722+01:00Excellent solution! Thank you very much!Excellent solution! Thank you very much!Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-84622269905073321332013-08-18T22:29:44.010+02:002013-08-18T22:29:44.010+02:00I was a bit shocked by the performance of the clas...I was a bit shocked by the performance of the class map. I'm going to see if I can get a chance to informally reproduce it, and also check if the same holds true for some of my other symfony stack projects.<br /><br />Also, Cymfony sounds exciting... have you released anything?jxnhttps://www.blogger.com/profile/02583794266862364944noreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-77719834019776471502013-07-04T06:33:39.272+02:002013-07-04T06:33:39.272+02:00Can you post the code you used to do this?Can you post the code you used to do this?Joshhttps://www.blogger.com/profile/11173872246949711723noreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-74500703336765655182013-03-11T11:19:13.801+01:002013-03-11T11:19:13.801+01:00@Jordi:
Using fresh content (non cached HTML):
Wi...@Jordi:<br />Using fresh content (non cached HTML):<br /><br />With APC[1]:<br />stock: 30.94 reqs/sec (avg: 32.320 ms)<br />removing loadClassCache(): 27.53 reqs/sec (avg: 36.324 ms)<br />(restoring loadClassCache() as it seems interesting for non cached pages)<br />activating ApcClassLoader: 35.52 reqs/sec (avg: 28.152 ms)<br />Registering the whole classmap in APC as suggested by Anonymous: 30.95 reqs/sec (avg: 32.307 ms)<br />Using Composer --optimize: 32.26 reqs/sec (avg: 31.000 ms)<br />Using PSR-0 tree: 35.50 reqs/sec (avg: 28.166 ms)<br /><br />ApcClassLoader seems a very little faster than the PSR-0 tree but I would say they would be about the same<br /><br />With Zend Optimizer+[2]:<br />stock: 29.21 reqs/sec (avg: 34.238 ms)<br />removing loadClassCache(): 27.75 reqs/sec (avg: 36.042 ms)<br />(restoring loadClassCache() as it seems interesting for non cached pages)<br />Using Composer --optimize: 30.67 reqs/sec (avg: 32.608 ms)<br />Using PSR-0 tree: 32.69 reqs/sec (avg: 30.590 ms)<br /><br />[1] APC settings:<br />apc.max_file_size=5M<br />apc.shm_segments=4<br /><br />[2] ZO+ settings:<br />zend_optimizerplus.memory_consumption=256<br />zend_optimizerplus.interned_strings_buffer=16<br />zend_optimizerplus.max_accelerated_files=4000<br />zend_optimizerplus.revalidate_freq=0<br />zend_optimizerplus.fast_shutdown=1<br />zend_optimizerplus.enable_cli=1<br />zend_optimizerplus.save_comments=0Patrick Allaerthttps://www.blogger.com/profile/08540817341697816823noreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-57394777014447309072013-03-11T09:09:50.766+01:002013-03-11T09:09:50.766+01:00For those who have asked, here is a detailed artic...For those who have asked, here is a detailed article containing the benchmarks: http://patrickallaert.blogspot.be/2013/03/benchmarking-composer-autoloading.htmlPatrick Allaerthttps://www.blogger.com/profile/08540817341697816823noreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-84825295702346394462013-03-08T22:34:23.500+01:002013-03-08T22:34:23.500+01:00Thank you for your efforts.
Could you also repeat...Thank you for your efforts.<br /><br />Could you also repeat the tests by adding the following patches?<br />https://github.com/symfony/symfony/pull/7071<br />and<br />https://github.com/symfony/symfony/pull/7064<br /><br />Also, if you look on the last patch, it should help you to add some classes from ezPublish as well to the cache that Symfony2 creates (ofc, don't add too many classes).<br /><br />Best regards.Florinhttps://www.blogger.com/profile/12549373005589656870noreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-12565186578315000872013-03-08T21:17:34.074+01:002013-03-08T21:17:34.074+01:00Cool data! I agree that the mass-concat-file appr...Cool data! I agree that the mass-concat-file approach requires proper tuning, and that tuning is probably per-install-specific.<br /><br />Another advantage of a unified PSR-0 tree for autoloading is that it reduces the number of directories to go through for developers to find a class. The Drupal community has been struggling with that for a while. Symlinks would address both problems.<br /><br />"Drop the expensive stuff to C" is one of the key advantages of autoloading and cleanly injected code. I'm a bit surprised that the Bags are so expensive. They may have optimization opportunity on their own. Worth investigating. I suspect the EventDispatcher would be another good C-ification target. Especially the ContainerAwareEventDispatcher has a ton of stack calls for every dispatch. IMO it needs to be compiled like the Container, but C-ifying it is an interesting idea, too.Larry Garfieldhttps://www.blogger.com/profile/14297333298035962476noreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-6925442877344436712013-03-08T20:57:41.123+01:002013-03-08T20:57:41.123+01:00I wonder how things would look if you cache the cl...I wonder how things would look if you cache the class map array insure the APC user cache which is optimized for fetching arrays. <br /><br />regards<br />LukasAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-29630983419933871342013-03-08T20:27:53.324+01:002013-03-08T20:27:53.324+01:00Very interesting results, but I think that a few t...Very interesting results, but I think that a few things are missing to properly test the classmap vs PSR-0 approach in Composer:<br /><br />- What about testing a real page view (not pre-cached html)? The current runs you showed don't load many classes so they probably don't offset the cost of loading the classmap.<br />- Could you do the test on 5.5 with O+ enabled? Curious to see if it handles loading the huge classmap better than APC (who knows).<br />Jordi Boggianohttp://seld.be/noreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-13761386231980341122013-02-14T18:19:45.943+01:002013-02-14T18:19:45.943+01:00@Stof: nope, haven't tried that since I know t...@Stof: nope, haven't tried that since I know the internals of PHP/APC a bit and know it wouldn't make any difference. The problem being that if you have an array containing thousands of entries, this array needs to be recreated in PHP userland space at every requests, regardless of where it comes from.Patrick Allaerthttps://www.blogger.com/profile/08540817341697816823noreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-26414404472241404572013-02-14T17:31:17.430+01:002013-02-14T17:31:17.430+01:00@Patrick did you tried to put the Composer classma...@Patrick did you tried to put the Composer classmap in the user cache instead of only relying on the opcode cache when recreating it ? I'm not sure it may help, but it would be worth a try.Stofnoreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-49139131229979343352013-02-12T16:07:53.876+01:002013-02-12T16:07:53.876+01:001. Symlinks do work on windows, but when symlinkin...1. Symlinks do work on windows, but when symlinking directories you have to be root<br /><br />2. would it not just make sense to split a huge classmap file in many small chunks, one per included library? This way a project can include everything-and-the-kitchen-sink, but we could expect that most web pages would only load classes from a few libraries out of all the provided ones, thus reducing time spent and memory wastedAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-4333076189681371352013-01-29T14:24:56.507+01:002013-01-29T14:24:56.507+01:00@Patrick I am not saying that your proposal is bad...@Patrick I am not saying that your proposal is bad either. However I would like to see a more exhaustive benchmark of your solution vs out of box class loading, composer optimized class loading and APC (user cache) optimized loading.<br /><br />It seems a bit premature to me to ask the composer team to implement your solution w/o those performance data.<br /><br />I think (stop me if I am wrong) that symlink would not work on windows. If the speed gain is limited, I would vote -1 for your solution.<br /><br />IMO, it would really be great if you can detail your exact test environment and provide the additional perf data - I can't say what you are testing by reading your conclusion.<br /><br />As a side note you might want to investigate generating your symlinks as a composer script.victorhttps://www.blogger.com/profile/13248041387789729023noreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-14640750073053552312013-01-29T11:30:11.474+01:002013-01-29T11:30:11.474+01:00@victor: I'm fully aware of the ApcClassLoader...@victor: I'm fully aware of the ApcClassLoader since this is what we used in eZ Publish 5. The thing is that if you really have a big amount of classes (we have ~2600 classes), those methods are still not as fast as having a direct translation from namespace+classname to path+file that doesn't need any data fetching at all.<br />I'm not saying that this technique should always be used, but it's worth trying it when it comes to benchmark as it might make a difference on your project.Patrick Allaerthttps://www.blogger.com/profile/08540817341697816823noreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-44740291657541702832013-01-29T00:43:33.866+01:002013-01-29T00:43:33.866+01:00@Patrick: I was posting from my mobile, sorry for ...@Patrick: I was posting from my mobile, sorry for the lack of details.<br /><br />I have indeed referring to the APC loader, based on the user cache (not to be confused with the opcode cache), see https://github.com/vicb/symfony/blob/master/src/Symfony/Component/ClassLoader/ApcClassLoader.php<br /><br />You should also specify if you benchmarks have APC enabled or not to be complete.victorhttps://www.blogger.com/profile/13248041387789729023noreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-14168940927795710542013-01-28T22:59:48.315+01:002013-01-28T22:59:48.315+01:00@victor Of course APC speed up things, but still i...@victor Of course APC speed up things, but still if you use it (which is the first thing to do for any PHP application, performance wise) you will gain a lot with this technique in the case your classmap is big. Read carefully the post to see why, I'm actually mentioning APC in it.Patrick Allaerthttps://www.blogger.com/profile/08540817341697816823noreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-16363840982356576452013-01-28T19:54:40.667+01:002013-01-28T19:54:40.667+01:00The easy way exists, apcThe easy way exists, apcvictorhttps://www.blogger.com/profile/13248041387789729023noreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-24238184236551290052013-01-27T11:28:40.617+01:002013-01-27T11:28:40.617+01:00@Anonymous: Nope, I haven't tried it. I starte...@Anonymous: Nope, I haven't tried it. I started with strpos() since it is faster than preg_match(), but since there is several strpos(), it might be that a unique preg_match() is faster. I would say that it depends very much of the order you select. If using a serie of strpos() like I did, be sure to have the strpos() matching the most frequent used namespaces at the top and finish with the less frequently used at the bottom to minimize the number of strpos() called.Patrick Allaerthttps://www.blogger.com/profile/08540817341697816823noreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-43161357160962839672013-01-26T19:05:46.689+01:002013-01-26T19:05:46.689+01:00Did you try using a preg_match instead of strpos t...Did you try using a preg_match instead of strpos to see if there is any further speed difference?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-188407679581479282013-01-26T15:01:27.614+01:002013-01-26T15:01:27.614+01:00@lsmith: I don't really see how this could lea...@lsmith: I don't really see how this could lead to a problem. This is only used for autoloading purpose. The entire setup made by Composer is preserved.<br />Note that files not respecting the PSR-0 standard inside those directories can still be loaded by additional autoloaders by chaining them. It's perfectly valid to still use Composers' autoload after the one I propose.Patrick Allaerthttps://www.blogger.com/profile/08540817341697816823noreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-90446511189527786922013-01-26T13:57:02.700+01:002013-01-26T13:57:02.700+01:00You cannot assume that you can just symlink direct...You cannot assume that you can just symlink directories, since in theory packages can place an individual file into overlapping subnamespaces, but I guess this just means that the symlink process needs to be smart enough to deal with that.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-7449668645955818675.post-12113574483477200652012-12-30T00:44:33.572+01:002012-12-30T00:44:33.572+01:00Would you really hack the first example or second!...Would you really hack the first example or second!! Maybe not that easy.<br />Good tut thoughAnonymousnoreply@blogger.com