Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests without any checks [C standard test] #323

Closed
belous-dp opened this issue Jul 14, 2022 · 1 comment
Closed

Tests without any checks [C standard test] #323

belous-dp opened this issue Jul 14, 2022 · 1 comment
Assignees
Labels
bug Something isn't working verified Bug fix is verified

Comments

@belous-dp
Copy link
Collaborator

belous-dp commented Jul 14, 2022

Description
Tests generated for functions with struct as return value don't contain assertions (EXPECT_, ASSERT_).

That happens only with "verbose formatting" disabled.

To Reproduce
Steps to reproduce the behavior:

  1. Set "Verbose formatting" setting disabled
  2. Open a file floating_point.c in the 'c-example' project
  3. Generate tests for the function fp_array:
struct FParray {
    float data[2];
};

struct FParray fp_array(int a) {
    if (a < 0) {
        struct FParray res = {{1.23, 3.21}};
        return res;
    }
    struct FParray res = {{12.3, 32.1}};
    return res;
}

Expected behavior
Tests are supposed to contain gtest assertions like EXPECT_, ASSERT_. For example, with the above source code I want to get tests like that:

TEST(regression, fp_array_test_1)
{
    struct FParray actual = fp_array(0);
    struct FParray expected = {
        .data = {1.230000e+01, 3.210000e+01}};
    EXPECT_EQ(sizeof(expected.data), sizeof(actual.data));
    size_t n = 0;
    if (sizeof(expected.data) > 0) {
        EXPECT_EQ(sizeof(expected.data) / sizeof(expected.data[0]), sizeof(actual.data) / sizeof(actual.data[0]));
        n = sizeof(expected.data) / sizeof(expected.data[0]);
    }
    for (int i = 0; i < n; i ++) {
        EXPECT_NEAR(expected.data[i], actual.data[i], utbot_abs_error);
    }
}

TEST(regression, fp_array_test_2) { ... }

Actual behavior
Instead, the generated tests do not contain any checks:

TEST(regression, fp_array_test_1)
{
    struct FParray actual = fp_array(0);
    struct FParray expected = {
        .data = {1.230000e+01, 3.210000e+01}};
}

TEST(regression, fp_array_test_2)
{
    struct FParray actual = fp_array(-1);
    struct FParray expected = {
        .data = {1.230000e+00, 3.210000e+00}};
}

Behaviour with "verbose formatting" on
Generated tests:

TEST(regression, fp_array_test_1)
{
    // Construct input
    int a = 0;



    // Expected output
    struct FParray expected = {
        .data = {1.230000e+01, 3.210000e+01}};

    // Trigger the function
    struct FParray actual = fp_array(a);

    // Check results
    for (int it_2_0 = 0; it_2_0 < 2; it_2_0 ++) {
        EXPECT_NEAR(expected.data[it_2_0], actual.data[it_2_0], utbot_abs_error);
    }
}

TEST(regression, fp_array_test_2)
{
    // Construct input
    int a = -1;



    // Expected output
    struct FParray expected = {
        .data = {1.230000e+00, 3.210000e+00}};

    // Trigger the function
    struct FParray actual = fp_array(a);

    // Check results
    for (int it_3_0 = 0; it_3_0 < 2; it_3_0 ++) {
        EXPECT_NEAR(expected.data[it_3_0], actual.data[it_3_0], utbot_abs_error);
    }
}

Comments
The same thing happens with any function with a struct as a return value. Examples: simple_structs.c, complex_structs.c.

The same with unions:
Source code:

union MainUnion {
    union InnerUnion {
        union InInnerUnion {
            unsigned int u;
            long long l;
        };
        char c;
        union InInnerUnion ininner;
        short s;
    } inner;

    int x;
    // union InnerUnion inner;
    long long y;
};

union MainUnion union_as_return_type(int a) {
    if (a == 0) {
        union MainUnion res = {{.c='0'}};
        return res;
    }

    if (a == 1) {
        union MainUnion res = {{.ininner={.l=1}}};
        return res;
    }

    union MainUnion res = {.x=2};
    return res;
}

Generated tests ("verbose formatting off):

TEST(regression, union_as_return_type_test_1)
{
    union MainUnion actual = union_as_return_type(4);
    union MainUnion expected = from_bytes<MainUnion>({2, 0, 0, 0, 0, 0, 0, 0});
}

TEST(regression, union_as_return_type_test_2)
{
    union MainUnion actual = union_as_return_type(1);
    union MainUnion expected = from_bytes<MainUnion>({1, 0, 0, 0, 0, 0, 0, 0});
}

TEST(regression, union_as_return_type_test_3)
{
    union MainUnion actual = union_as_return_type(0);
    union MainUnion expected = from_bytes<MainUnion>({48, 0, 0, 0, 0, 0, 0, 0});
}

Maybe unions can be compared via memcmp.

Environment
UTBotCpp version 2022.7.0, tested locally using Docker+CLion+VSCode and tested online using utbot.

@belous-dp belous-dp added the bug Something isn't working label Jul 14, 2022
@korifey korifey moved this to Todo in UTBot C/C++ Jul 14, 2022
@belous-dp belous-dp changed the title Tests without the assertions Tests without the assertions [C standard test] Jul 15, 2022
@belous-dp belous-dp changed the title Tests without the assertions [C standard test] Tests without the assertions in short formatting [C standard test] Jul 18, 2022
@Lana243 Lana243 changed the title Tests without the assertions in short formatting [C standard test] Tests without any checks [C standard test] Jul 18, 2022
@ladisgin ladisgin added the need to check Check if the issue can be reproduced in last version label Nov 15, 2022
@tyuldashev tyuldashev self-assigned this Nov 16, 2022
@tyuldashev
Copy link
Collaborator

Not reproducible anymore. Checked on v2022.11.298.

Repository owner moved this from Todo to Done in UTBot C/C++ Nov 17, 2022
@tyuldashev tyuldashev added verified Bug fix is verified and removed need to check Check if the issue can be reproduced in last version labels Nov 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working verified Bug fix is verified
Projects
Status: Done
Development

No branches or pull requests

4 participants