Comments on: Measuring performance http://robarnold.org/measuring-performance/ witty tagline Tue, 07 Sep 2010 15:18:56 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Rob Arnold http://robarnold.org/measuring-performance/comment-page-1/#comment-6701 Rob Arnold Thu, 14 Jan 2010 02:47:37 +0000 http://robarnold.org/?p=33#comment-6701 That methods looks neat, but I'm not sure how well it would work in a virtualized or headless environment. Also it's not clear what API requirements they have for using it - I'm not sure if you can get the vsync notifications on Vista with the DWM enabled. That methods looks neat, but I’m not sure how well it would work in a virtualized or headless environment. Also it’s not clear what API requirements they have for using it – I’m not sure if you can get the vsync notifications on Vista with the DWM enabled.

]]> By: James http://robarnold.org/measuring-performance/comment-page-1/#comment-3807 James Wed, 04 Nov 2009 22:54:12 +0000 http://robarnold.org/?p=33#comment-3807 You may be interested in the methods used by the PsychToolbox development team for highly accurate time durations. They cross reference beam positions with the system time stamp. Here's a doc discussion on the method: http://docs.psychtoolbox.org/BeampositionQueries They claim accuracies in the 100 microsecond range and lower on OSX and Windows. You may be interested in the methods used by the PsychToolbox development team for highly accurate time durations. They cross reference beam positions with the system time stamp. Here’s a doc discussion on the method:

http://docs.psychtoolbox.org/BeampositionQueries

They claim accuracies in the 100 microsecond range and lower on OSX and Windows.

]]> By: Кофеваркин Волк http://robarnold.org/measuring-performance/comment-page-1/#comment-537 Кофеваркин Волк Wed, 20 May 2009 09:12:21 +0000 http://robarnold.org/?p=33#comment-537 Хороший пост по этой теме нечасто встретишь, немногие хотят копать так глубоко, мне понравилось ваше мнение Хороший пост по этой теме нечасто встретишь, немногие хотят копать так глубоко, мне понравилось ваше мнение

]]> By: Пуфиков Антоныч http://robarnold.org/measuring-performance/comment-page-1/#comment-534 Пуфиков Антоныч Tue, 19 May 2009 15:30:21 +0000 http://robarnold.org/?p=33#comment-534 Спасибо за грамотное изложение темы, я бы добавил от себя немного, но я не глубокий спец в этой тематике =( Спасибо за грамотное изложение темы, я бы добавил от себя немного, но я не глубокий спец в этой тематике =(

]]> By: Rob Arnold http://robarnold.org/measuring-performance/comment-page-1/#comment-106 Rob Arnold Mon, 22 Dec 2008 19:06:20 +0000 http://robarnold.org/?p=33#comment-106 Well the Unix time format doesn't account for leap seconds which is the format returned by time and gettimeofday (in its timeval struct). I'm not sure how the functions that convert from a time_t to a struct tm* handle leap seconds. If you look at Wikipedia's article on Unix time, they have a nice table where they show that the leap second does not have a unique value in unix time. For performance measurements, it's much more convenient to use unix time since subtraction on full dates is non-trivial. So perhaps the struct tm* returning functions are correct which is what your clock would use, but you will get duplicate time_t values from time and gettimeofday which is what Date.now() uses. QueryPerformanceCounter can give you microseconds (and maybe even nanoseconds), but Date.now() calls PRMJ_Now which returns and then does an integer divide by 1000 to get milliseconds. I'm not sure how accurate those numbers would be though since there are a lot of factors that can affect performance at that level Well the Unix time format doesn’t account for leap seconds which is the format returned by time and gettimeofday (in its timeval struct). I’m not sure how the functions that convert from a time_t to a struct tm* handle leap seconds. If you look at Wikipedia’s article on Unix time, they have a nice table where they show that the leap second does not have a unique value in unix time. For performance measurements, it’s much more convenient to use unix time since subtraction on full dates is non-trivial. So perhaps the struct tm* returning functions are correct which is what your clock would use, but you will get duplicate time_t values from time and gettimeofday which is what Date.now() uses.

QueryPerformanceCounter can give you microseconds (and maybe even nanoseconds), but Date.now() calls PRMJ_Now which returns and then does an integer divide by 1000 to get milliseconds. I’m not sure how accurate those numbers would be though since there are a lot of factors that can affect performance at that level

]]> By: Rob Arnold http://robarnold.org/measuring-performance/comment-page-1/#comment-105 Rob Arnold Mon, 22 Dec 2008 17:06:35 +0000 http://robarnold.org/?p=33#comment-105 I haven't checked what gettimeofday does on the POSIX subsystem, but I'd imagine that it suffers from the same problem as GetSystemTimeAsFileTime. Ditto for Visual Studio's CRT. I haven’t checked what gettimeofday does on the POSIX subsystem, but I’d imagine that it suffers from the same problem as GetSystemTimeAsFileTime. Ditto for Visual Studio’s CRT.

]]> By: Mook http://robarnold.org/measuring-performance/comment-page-1/#comment-104 Mook Mon, 22 Dec 2008 16:48:38 +0000 http://robarnold.org/?p=33#comment-104 For things like Talos (where the goal is to run quickly, in a browser-specific fashion), is there any reason to not expose possibly better APIs via an extra component and just fork the tests? Yes, this might affect the results some, but I don't think it would be significant, especially if all runs were made with the same? For things like Talos (where the goal is to run quickly, in a browser-specific fashion), is there any reason to not expose possibly better APIs via an extra component and just fork the tests? Yes, this might affect the results some, but I don’t think it would be significant, especially if all runs were made with the same?

]]> By: Robert Kaiser http://robarnold.org/measuring-performance/comment-page-1/#comment-103 Robert Kaiser Mon, 22 Dec 2008 13:17:36 +0000 http://robarnold.org/?p=33#comment-103 I read some interesting discussion on high-resolution time measurement for Linux performance tests back when "dynamic tick" support on Linux was quite new, as suddenly people couldn't just take the number of ticks as a consistent time measurement any more ;-) Unfortunately, as far as I remember, they also had no really good solution there, though I still wonder if we could tap into what the POSIX time command does, as that one nicely can separate real and user/system time somehow, which reduces the influence of other processes running on the machine. Not sure though if Windows provides something like that. I read some interesting discussion on high-resolution time measurement for Linux performance tests back when “dynamic tick” support on Linux was quite new, as suddenly people couldn’t just take the number of ticks as a consistent time measurement any more ;-)
Unfortunately, as far as I remember, they also had no really good solution there, though I still wonder if we could tap into what the POSIX time command does, as that one nicely can separate real and user/system time somehow, which reduces the influence of other processes running on the machine.
Not sure though if Windows provides something like that.

]]> By: Anonymous http://robarnold.org/measuring-performance/comment-page-1/#comment-102 Anonymous Mon, 22 Dec 2008 09:57:20 +0000 http://robarnold.org/?p=33#comment-102 Linux does support leap seconds. When a leap second occurs, a given minute will have seconds ranging from 0-60 instead of 0-59. Also, Linux doesn't just support millisecond accuracy via gettimeofday; gettimeofday returns time in microseconds. Furthermore, if you need better resolution than that you can use clock_gettime, which returns time in nanoseconds. Both of those exist in POSIX, though not all systems have the latter. Linux actually keeps time as accurately as the clock sources you have available, all the way down to the nanosecond if possible. With the Linux -rt patches for real-time, you can readily attain response times measured in small numbers of microseconds. Linux does support leap seconds. When a leap second occurs, a given minute will have seconds ranging from 0-60 instead of 0-59.

Also, Linux doesn’t just support millisecond accuracy via gettimeofday; gettimeofday returns time in microseconds. Furthermore, if you need better resolution than that you can use clock_gettime, which returns time in nanoseconds. Both of those exist in POSIX, though not all systems have the latter. Linux actually keeps time as accurately as the clock sources you have available, all the way down to the nanosecond if possible. With the Linux -rt patches for real-time, you can readily attain response times measured in small numbers of microseconds.

]]> By: Justin Dolske http://robarnold.org/measuring-performance/comment-page-1/#comment-101 Justin Dolske Mon, 22 Dec 2008 06:29:37 +0000 http://robarnold.org/?p=33#comment-101 So, in conclusion, Windows doesn't know what time it is, but Linux and OS X do? :-) Out of curiosity, doesn't Windows have a POSIX compatibility thing? Does its gettimeofday() also suffer from granularity problems, or is it just full of fail? So, in conclusion, Windows doesn’t know what time it is, but Linux and OS X do? :-)

Out of curiosity, doesn’t Windows have a POSIX compatibility thing? Does its gettimeofday() also suffer from granularity problems, or is it just full of fail?

]]>