Results 1 to 9 of 9

Thread: RBEST not working in some matrices

  1. #1
    Junior Member
    Join Date
    Mar 2018
    Posts
    7

    RBEST not working in some matrices

    Hello,

    The RBEST Fortran IMSL routine fails to give a best regression for some matrices. I attach one of them. I included inline the code that reads it, calls COV and then RBEST. The print set with IPRINT 1 gets truncated and the output arrays don't have the answer.

    What's the recommended approach in this cases?

    Thanks

    Code:

    program testrbest
    !use msimsl
    implicit real*8 (a-h,o-z)

    INTEGER LDCOEF, LDCOV, LDX, NBEST, NGOOD, NSIZE, NTBEST, NVAR
    PARAMETER (LDX=50, NBEST=1, NGOOD=10, NVAR=13, &
    LDCOEF=NBEST*(NVAR-1)*NVAR/2, LDCOV=NVAR, &
    NSIZE=NVAR-1, NTBEST=NBEST*(NVAR-1))
    INTEGER ICOEFX(NTBEST+1), ICOPT, ICRIT, ICRITX(NSIZE+1), &
    IFRQ, INCD(1,1), INDVAR(NGOOD*NSIZE*(NSIZE+1)/2), &
    IPRINT, IVARX(NSIZE+1), IWT, MOPT, NMISS, NOBS, NROW, &
    NVAR1
    dimension COEF(LDCOEF,13), COV(LDCOV,NVAR), CRIT(NGOOD*NSIZE), &
    X(LDX,NVAR), XMEAN(NVAR)

    CHARACTER(LEN=500) :: file_path
    integer file_id


    NROW = LDX
    ! Open matrix file
    file_id = 1
    file_path="matrix.txt"
    open(file_id,file=file_path)
    ! Read matrix
    do i=1,NROW
    read(file_id,*) (X(i,j), j = 1, NVAR)
    end do

    ! Call to get covariance matrix
    IFRQ = 0
    IWT = 0
    MOPT = 0
    ICOPT = 1
    CALL DCORVC (0, NROW, NVAR, X, LDX, IFRQ, IWT, MOPT, ICOPT, &
    XMEAN, COV, LDCOV, INCD, 1, NOBS, NMISS, SUMWT)
    ICRIT = 2
    IPRINT = 1
    ! Call to get best regression
    CALL DRBEST (NVAR, COV, LDCOV, NOBS, ICRIT, NBEST, NGOOD, IPRINT, &
    ICRITX, CRIT, IVARX, INDVAR, ICOEFX, COEF, LDCOEF)
    stop
    END
    Attached Files Attached Files

  2. #2
    Administrator
    Join Date
    Feb 2008
    Posts
    23
    With the current release of IMSL Fortran library (Version 2018), I am not able to replicate the described behavior. So the problem you might be seeing may be fixed in a newer release of the software and I would recommend upgrading to the latest version. If you have a current valid license, you can request an upgrade here:
    https://www.roguewave.com/help-support/update-request

    If you do not have current valid license, then you would need to contact Rogue Wave Sales:
    https://www.roguewave.com/company/contact-us

    Jeremy
    Jeremy Dean
    Rogue Wave Software
    Technical Support
    support@roguewave.com

  3. #3
    Junior Member
    Join Date
    Mar 2018
    Posts
    7
    How do I check my IMSL version?

    I attach the output I get for the matrix from the first post. It never gets to the part of:

    Best Regression with X variable(s) (Adjusted R-squared)
    Variable Coefficient Standard Error t-statistic p-value
    ...

    Could you show me what output you get with this matrix?

    Thanks
    Attached Files Attached Files

  4. #4
    Administrator
    Join Date
    Feb 2008
    Posts
    23
    Here is the output from IMSL Fortran Numeric Library version 2018.0.
    Attached Files Attached Files
    Jeremy Dean
    Rogue Wave Software
    Technical Support
    support@roguewave.com

  5. #5
    Senior Member mecej4's Avatar
    Join Date
    Dec 2009
    Posts
    127
    Ikno, you did not state which version of IMSL you used, but here is a work-around that gave me the same results as Dean's with FNL 4 as well as FNL 7.0: instead of setting NBEST=1, set NBEST=2 and discard the second best results.

    You can find the IMSL version number in a number of ways.

    (i) The directory where IMSL is installed may itself contain the version number in the path.

    (ii) The README.TXT in the IMSL directory may have the information.

    (iii) Look for VERSL in the IMSL Stat manual. Take the example code for VERSL and run it. It will print out the version number.
    Last edited by mecej4; 03-13-2018 at 08:34 PM.

  6. #6
    Junior Member
    Join Date
    Mar 2018
    Posts
    7
    Thanks for the help, mecej4.

    This is the first line in the IMSL readme:
    README files for Fortran Numerical Library 7.1.0 are located in the subdirectory

    Therefore, I have version 7.1.0. Should I update? I think that's the last version.

    I found another workaround by normalizing the matrix beforehand. The output of calling RBEST with the normalized matrix is compatible with dean's output.

    I tried your workaround of setting NBEST = 2 and in some cases it changes the best solution. For example, when swapping the first column of the matrix with the last one, using NBEST=1 the best solution is:
    0 1 1 1 0 1 1 1 1 1 0 1 1
    With NBEST=2:
    0 1 0 1 0 1 1 1 1 1 0 1 1

    That is, with NBEST=1, the best solution includes column 3 while NBEST=2 doesn't.

    I don't know if it's related or not, but I installed it in another machine and I'm getting different results. Not even when normalizing the matrix beforehand in this new machine I get answers with NBEST=1.

  7. #7
    Junior Member
    Join Date
    Mar 2018
    Posts
    7
    I think the difference between results comes from different hardware and not from different software. In the new machine I get the same output as dean's from the matrix of the first post of this thread. That is, without normalizing anything.

    Nevertheless, I get no output from the matrix attached to this post using the new machine, while I get a valid output in the old machine. I think that you won't get any output either. Please try with the attached matrix in this post. This matrix is just the original but with the 3rd column put as the last one.


    It's the same Intel and IMSL version installed in both machines and I get different outputs. I attach both outputs.

    I added the following lines to the code from the first post to print the results according to ICOEFX:

    write(*,*) "Last row in COEF:"
    write(*,*) ICOEFX(1)
    write(*,*) "Last row in COEF:"
    write(*,*) ICOEFX(2)-1
    Attached Files Attached Files
    Last edited by ikno200; 04-03-2018 at 02:32 PM. Reason: redundant comments in code excerpt

  8. #8
    Junior Member
    Join Date
    Mar 2018
    Posts
    7
    If anyone get's stuck with this problem in the future, just choose R2 instead of Adjusted R2 by setting "ICRIT = 1" instead of "ICRIT = 2". The bug seems to be in the Adjusted R2 calculation but not in the base R2 calculation.

  9. #9
    Junior Member
    Join Date
    Mar 2018
    Posts
    7
    Quote Originally Posted by chetansolanki894 View Post
    If anyone get's stuck with this problem in the future, just choose R2 instead of Adjusted R2 by setting "ICRIT = 1" instead of "ICRIT = 2". The bug seems to be in the Adjusted R2 calculation but not in the base R2 calculation.

    Thank for this
    Yes, but take into account that these are different metrics. You would need to calculate Adjusted R2 by hand using the R2 results. Also, the Adjusted R2 may use heuristics using intermediate values that are "lost" if one does R2 from RBEST and Adjusted R2 by hand.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •