Return-Path: X-Original-To: apmail-incubator-flex-dev-archive@minotaur.apache.org Delivered-To: apmail-incubator-flex-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 353D4D3AF for ; Sun, 9 Dec 2012 23:28:23 +0000 (UTC) Received: (qmail 32409 invoked by uid 500); 9 Dec 2012 23:28:22 -0000 Delivered-To: apmail-incubator-flex-dev-archive@incubator.apache.org Received: (qmail 32371 invoked by uid 500); 9 Dec 2012 23:28:22 -0000 Mailing-List: contact flex-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: flex-dev@incubator.apache.org Delivered-To: mailing list flex-dev@incubator.apache.org Received: (qmail 32361 invoked by uid 99); 9 Dec 2012 23:28:22 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 09 Dec 2012 23:28:22 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of webdoublefx@hotmail.com designates 65.55.111.85 as permitted sender) Received: from [65.55.111.85] (HELO blu0-omc2-s10.blu0.hotmail.com) (65.55.111.85) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 09 Dec 2012 23:28:13 +0000 Received: from BLU162-DS8 ([65.55.111.73]) by blu0-omc2-s10.blu0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Sun, 9 Dec 2012 15:27:51 -0800 X-Originating-IP: [82.247.153.175] X-EIP: [YN0m2sZFJHHQZEbdok3kpyAsApea+OjH] X-Originating-Email: [webdoublefx@hotmail.com] Message-ID: From: =?iso-8859-1?Q?Fr=E9d=E9ric_THOMAS?= To: References: In-Reply-To: Subject: Re: [Mustella] 90 tests failed Date: Mon, 10 Dec 2012 00:27:48 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 8bit X-Priority: 3 X-MSMail-Priority: Normal Importance: Normal X-Mailer: Microsoft Windows Live Mail 16.4.3505.912 x-mimeole: Produced By Microsoft MimeOLE V16.4.3505.912 X-OriginalArrivalTime: 09 Dec 2012 23:27:51.0649 (UTC) FILETIME=[CEB27D10:01CDD664] X-Virus-Checked: Checked by ClamAV on apache.org Hi Nick, Thank you for clarifying these points :) - Fred. -----Message d'origine----- From: Nicholas Kwiatkowski Sent: Sunday, December 09, 2012 9:18 PM To: flex-dev@incubator.apache.org Subject: Re: [Mustella] 90 tests failed Usually the goal is to /not/ change the image that is generated. If it changes, that means it has the potential to break somebody's code, or some project relying on the behavior that existed before. Mustella, as you are aware is a functional testing suite, so in addition to checking the validity of functions, etc. it also takes screen-shots of the sample applications and compares them with the ones in it's 'database'. If they differ then the test fails. There should be very little need to regenerate the images in general. The ones that are checked in should be pretty agnostic, and should already cover all the components and how they are supposed to behave on every platform. The reason why the charts and ADG fail is because the images that were submitted with the tests were either wrong, or there is something that is not OS/FP specific that is being checked (shouldn't happen). I was generating my own images for those components based on the 4.6 on my local machine because tests with images didn't exist. I needed to test to make sure that the patches I was putting in that dealt with the charts didn't break. I saw somebody chime in saying they were going to fix those tests, but I forget who. I'm sure proper images will be checked in and your tests should pass at that point. -Nick On Sun, Dec 9, 2012 at 10:50 AM, Fr�d�ric THOMAS wrote: > Hi Nick, > > Ok, I guess I see your point, it's just when you change the code of the > framework, it might makes differences in the generated images, right ? > So, it would mean that AdvancedDataGrid and Charts failed because of other > changes in the framework or I missed somthing in the reason you regenerate > the images against the 4.6 and and rerun tests against the current > framework ? > > (Sorry for asking to dig into details but I'm discovering Mustella and I > might miss how things might impact others) > > > - Fred > > -----Message d'origine----- From: Nicholas Kwiatkowski > Sent: Sunday, December 09, 2012 4:29 PM > > To: flex-dev@incubator.apache.org > Subject: Re: [Mustella] 90 tests failed > > Well, the point is to test any code that you may have broke. 4.6 is > supposed to be the baseline, and if I break something that was working in > 4.6, then I need to know about it. > > -Nick > > On Sun, Dec 9, 2012 at 10:26 AM, Fr�d�ric THOMAS > **wrote: > > Hi Nick, >> >> Tell me if I'm wrong, but regenerating them against the 4.6 is the same >> than the 4.8 as they both use the same swfVersion, right ? >> At the end, you just regenerate them against your own system, right ? >> >> - Fred >> >> -----Message d'origine----- From: Nicholas Kwiatkowski >> Sent: Sunday, December 09, 2012 2:13 PM >> >> To: flex-dev@incubator.apache.org >> Subject: Re: [Mustella] 90 tests failed >> >> One quick test I do is to generate the base-line images running mustella >> against 4.6, and then run mustella against the current tests that were >> failing. >> >> That should at least work for the image comparison issues you are >> seeing.. >> That is at least what I was doing for the charting components before the >> official tests were donated. >> >> -Nick >> >> On Sun, Dec 9, 2012 at 2:11 AM, Alex Harui wrote: >> >> I will investigate the test failures over the next few days. There are >> >>> two >>> sets of failures that are troubling that we want to make sure isn't in >>> 4.9. >>> The rest might be flaws in the tests themselves. >>> >>> If you get the same failures with or without the patch, then it is ok to >>> accept the patch. I know it isn't the safest policy, but I think it is >>> good >>> enough, especially to not discourage community contributions. >>> >>> >>> On 12/8/12 10:50 PM, "Fr�d�ric THOMAS" wrote: >>> >>> > Hi Alex, >>> > >>> > What then for the failed tests not relative to AdvancedDataGrid and >>> Charts >>> > ? >>> > >>> > I've got no clue how to fix them and some are relative to CSS, except >>> > I >>> > wanted to run the test suite against a patch which modify >>> > CSSCondition, >>> does >>> > it worth to test it anyway or I have to wait for someone fix the >>> > failed >>> > tests before ? >>> > >>> > - Fred >>> > >>> > -----Message d'origine----- >>> > From: Alex Harui >>> > Sent: Sunday, December 09, 2012 6:25 AM >>> > To: flex-dev@incubator.apache.org >>> > Subject: Re: [Mustella] 90 tests failed >>> > >>> > I would ignore AdvancedDataGrid and Charts. The tests are newly > >>> donated >>> and >>> > I'm not sure they have been proven to work on both platforms. >>> > >>> > I ran the full suite last night and saw the same other errors. >>> > >>> > >>> > On 12/8/12 8:54 PM, "Fr�d�ric THOMAS" wrote: >>> > >>> >> After a closer look, they all failed comparing bitmaps except for the >>> >> three >>> >> last where it's a locale resources problem. >>> >> >>> >> - Fred >>> >> >>> >> -----Message d'origine----- >>> >> From: Fr�d�ric THOMAS >>> >> Sent: Sunday, December 09, 2012 5:36 AM >>> >> To: flex-dev@incubator.apache.org >>> >> Subject: [Mustella] 90 tests failed >>> >> >>> >> Hi, >>> >> >>> >> Can someone confirm he recently ran the all test suite with no failed >>> >> tests >>> >> ? >>> >> >>> >> I just ran it and had 90 tests failed[1] >>> >> After a quick look at the advanceddatagrid ones, it appear that�s due >>> >> to >>> >> the >>> >> sort separator comparing bitmaps and the edges of the glyphs too. >>> >> >>> >> - Fred >>> >> >>> >> [1] >>> >> >>> >>> Managers/StyleManager/****AdvancedCSS/classSelectors/** >>> AdvancedCSS_ClassSelectors_R>> >>> u >>> >> ntimeCSS >>> >> id_classSelector_basic_style >>> >> >>> >>> Managers/StyleManager/****AdvancedCSS/classSelectors/** >>> AdvancedCSS_ClassSelectors_R>> >>> u >>> >> ntimeCSS >>> >> id_classSelector_basic_style_****neg >>> >> >>> >>> Managers/StyleManager/****AdvancedCSS/idSelectors/** >>> AdvancedCSS_IDSelectors_****Runtime>> >>> C >>> >> SS >>> >> id_multipleTypeSelector_style >>> >> Managers/ToolTipManager/****Properties/ToolTipManager_**** >>> Properties_spark >>> >> Spark_ToolTipManager_property_****customToolTip_list >>> >> Managers/ToolTipManager/****Properties/ToolTipManager_**** >>> Properties_spark >>> >> Spark_ToolTipManager_property_****ToolTip_list_disabled >>> >> PerModuleStyles/BasicTests/****loadFonts loadFonts >>> >> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> columngroup >>> >> use_grouped_data_mirroring >>> >> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> columngroup >>> >> use_grouped_data >>> >> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> columngroup >>> >> set_editable_item >>> >> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> columngroup >>> >> set_group_item >>> >> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> columngroup >>> >> set_both_editable >>> >> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> columngroup >>> >> insert_columns >>> >> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> columngroup >>> >> insert_columngroup >>> >> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> columngroup >>> >> hierarchical_data >>> >> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> columngroup >>> >> Row_Count_with_ColumnGroups >>> >> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> columngroup >>> >> Formatting_with_column_groups >>> >> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> columngroup >>> >> Column_Span >>> >> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> columngroup >>> >> Header_Renderer >>> >> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> columngroup >>> >> Custom_Header_Renderer >>> >> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> columngroup >>> >> Visibility_of_columns >>> >> components/AdvancedDataGrid/****Properties/adg_mirroring_test >>> >> property_layoutDirection_ltr >>> >> components/AdvancedDataGrid/****Properties/adg_mirroring_test >>> >> property_layoutDirection_rtl >>> >> components/AdvancedDataGrid/****Properties/adg_mirroring_test >>> >> property_layoutDirection_****application >>> >> components/AdvancedDataGrid/****Properties/adg_mirroring_test >>> >> property_layoutDirection_****application_ltr >>> >> components/AdvancedDataGrid/****Properties/advanceddatagrid_**** >>> properties >>> >> datagrid_own_properties_****draggablecolumns_true >>> >> components/AdvancedDataGrid/****Properties/advanceddatagrid_**** >>> properties >>> >> datagrid_own_properties_****draggablecolumns_false >>> >> components/AdvancedDataGrid/****Properties/advanceddatagrid_**** >>> properties >>> >> datagrid_own_properties_****applygrouping >>> >> components/AdvancedDataGrid/****Properties/advanceddatagrid_**** >>> properties >>> >> datagrid_own_properties_****treeview >>> >> components/AdvancedDataGrid/****Properties/advanceddatagrid_**** >>> properties >>> >> datagrid_own_properties_****selectionMode_flat_single >>> >> components/AdvancedDataGrid/****Properties/advanceddatagrid_**** >>> properties >>> >> datagrid_own_properties_****selectionMode_flat_multiple >>> >> components/AdvancedDataGrid/****Properties/advanceddatagrid_**** >>> properties >>> >> datagrid_own_properties_****selectionMode_tree_singlecell >>> >> components/AdvancedDataGrid/****Properties/advanceddatagrid_**** >>> properties >>> >> datagrid_own_properties_****selectionMode_tree_multicells >>> >> components/AdvancedDataGrid/****Properties/advanceddatagrid_**** >>> properties >>> >> datagrid_own_properties_****selectionMode_tree_**** >>> displayitemsexpanded >>> >> >>> >>> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> properties_scrollcont>> >>> r >>> >> olbase >>> >> propert_vscrollposition >>> >> >>> >>> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> properties_scrollcont>> >>> r >>> >> olbase >>> >> propert_vscrollpolicy_off >>> >> >>> >>> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> properties_scrollcont>> >>> r >>> >> olbase >>> >> propert_vscrollpolicy_on >>> >> >>> >>> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> properties_scrollcont>> >>> r >>> >> olbase >>> >> propert_vscrollpolicy_auto >>> >> >>> >>> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> properties_scrollcont>> >>> r >>> >> olbase >>> >> propert_hscrollpolicy_on >>> >> >>> >>> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> properties_scrollcont>> >>> r >>> >> olbase >>> >> propert_hscrollpolicy_off >>> >> >>> >>> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> properties_uicomponen>> >>> t >>> >> propert_enabled_****withdisabledcolor >>> >> >>> >>> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> properties_uicomponen>> >>> t >>> >> propert_width >>> >> >>> >>> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> properties_uicomponen>> >>> t >>> >> propert_width_200 >>> >> >>> >>> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> properties_uicomponen>> >>> t >>> >> propert_width_30 >>> >> >>> >>> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> properties_uicomponen>> >>> t >>> >> propert_height >>> >> >>> >>> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> properties_uicomponen>> >>> t >>> >> propert_height_200 >>> >> >>> >>> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> properties_uicomponen>> >>> t >>> >> propert_height_30 >>> >> >>> >>> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> properties_uicomponen>> >>> t >>> >> propert_x_50 >>> >> >>> >>> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> properties_uicomponen>> >>> t >>> >> propert_x_minus10 >>> >> >>> >>> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> properties_uicomponen>> >>> t >>> >> propert_y_50 >>> >> >>> >>> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> properties_uicomponen>> >>> t >>> >> propert_y_minus10 >>> >> >>> >>> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> properties_uicomponen>> >>> t >>> >> propert_focusEnabled_true >>> >> >>> >>> components/AdvancedDataGrid/****Properties/advanceddatagrid_** >>> properties_uicomponen>> >>> t >>> >> propert_focusEnabled_false >>> >> components/AdvancedDataGrid/****Styles/advanceddatagrid_styles >>> >> style_headerColors >>> >> components/AdvancedDataGrid/****Styles/advanceddatagrid_styles >>> >> style_verticalGridLines >>> >> components/AdvancedDataGrid/****Styles/advanceddatagrid_** >>> styles_listbase >>> >> style_alternatingitemcolors >>> >> components/AdvancedDataGrid/****Styles/advanceddatagrid_** >>> styles_listbase >>> >> style_paddingBottom >>> >> components/AdvancedDataGrid/****Styles/advanceddatagrid_** >>> styles_listbase >>> >> style_paddingtop >>> >> components/AdvancedDataGrid/****Styles/advanceddatagrid_** >>> styles_listbase >>> >> style_paddingleft >>> >> components/AdvancedDataGrid/****Styles/advanceddatagrid_** >>> styles_listbase >>> >> style_paddingright >>> >> components/AdvancedDataGrid/****Styles/advanceddatagrid_** >>> styles_listbase >>> >> style_userollover >>> >> components/AdvancedDataGrid/****Styles/advanceddatagrid_** >>> styles_listbase >>> >> style_verticalalign_top >>> >> components/AdvancedDataGrid/****Styles/advanceddatagrid_** >>> styles_listbase >>> >> style_verticalalign_bottom >>> >> >>> components/AdvancedDataGrid/****Styles/advanceddatagrid_** >>> styles_scrollcontrolbase >>> >> style_borderSides_all >>> >> >>> components/AdvancedDataGrid/****Styles/advanceddatagrid_** >>> styles_scrollcontrolbase >>> >> style_borderSides_lr >>> >> >>> components/AdvancedDataGrid/****Styles/advanceddatagrid_** >>> styles_scrollcontrolbase >>> >> style_borderStyle_inset >>> >> >>> components/AdvancedDataGrid/****Styles/advanceddatagrid_** >>> styles_scrollcontrolbase >>> >> style_borderStyle_outset >>> >> >>> components/AdvancedDataGrid/****Styles/advanceddatagrid_** >>> styles_scrollcontrolbase >>> >> style_fontFamily_mxml >>> >> >>> components/AdvancedDataGrid/****Styles/advanceddatagrid_** >>> styles_scrollcontrolbase >>> >> style_fontFamily_AS >>> >> >>> components/AdvancedDataGrid/****Styles/advanceddatagrid_** >>> styles_scrollcontrolbase >>> >> style_fontWeight >>> >> >>> components/AdvancedDataGrid/****Styles/advanceddatagrid_** >>> styles_scrollcontrolbase >>> >> style_fontStyle >>> >> >>> components/AdvancedDataGrid/****Styles/advanceddatagrid_** >>> styles_scrollcontrolbase >>> >> style_fontSize >>> >> >>> components/AdvancedDataGrid/****Styles/advanceddatagrid_** >>> styles_scrollcontrolbase >>> >> style_dropShadowEnabled >>> >> >>> components/AdvancedDataGrid/****Styles/advanceddatagrid_** >>> styles_scrollcontrolbase >>> >> style_shadowDirection >>> >> >>> components/AdvancedDataGrid/****Styles/advanceddatagrid_** >>> styles_scrollcontrolbase >>> >> style_shadowDistance >>> >> >>> components/AdvancedDataGrid/****Styles/advanceddatagrid_** >>> styles_scrollcontrolbase >>> >> style_textDecoration >>> >> components/AdvancedDataGrid/****Properties/formatters >>> use_style_function1 >>> >> components/AdvancedDataGrid/****Properties/formatters >>> use_style_function2 >>> >> components/AdvancedDataGrid/****Properties/formatters >>> use_style_function3 >>> >> components/AdvancedDataGrid/****Properties/formatters >> >>> use_currency_formatter >>> >> components/AdvancedDataGrid/****Properties/formatters >>> use_number_formatter >>> >> components/AdvancedDataGrid/****Properties/formatters >>> use_phone_formatter >>> >> components/AdvancedDataGrid/****Properties/formatters >>> use_date_formatter >>> >> components/AdvancedDataGrid/****Properties/formatters >>> use_zipcode_formatter >>> >> components/AdvancedDataGrid/****Properties/formatters >>> use_grouped_view >>> >> components/AdvancedDataGrid/****Properties/formatters use_columnspan >>> >> components/Charts/Bubble/****Properties/BubbleProperties >>> >> BubbleSeries_multipleAxes_****disabledDays >>> >> gumbo/components/DropDownList/****Properties/DropDownList_** >>> Properties_Images >>> >> DropDownList_Properties_****imageTest2 >>> >> >>> resources/ResourceManager/****Methods/ResourceManager_** >>> Methods_loadResourceModule >>> >> RTL_Method_loadResourceModule_****SameLocaleSameBundle >>> >> >>> resources/ResourceManager/****Methods/ResourceManager_** >>> Methods_loadResourceModule >>> >> RTL_Method_loadResourceModule_****AdditionalLocale1Bundle >>> >> >>> resources/ResourceManager/****Methods/ResourceManager_** >>> Methods_loadResourceModule >>> >> RTL_Method_loadResourceModule_****AdditionalLocales3Bundles >>> >>> >> >>> >>> -- >>> Alex Harui >>> Flex SDK Team >>> Adobe Systems, Inc. >>> http://blogs.adobe.com/aharui >>> >>> >>> >>> >> >