Encode in 2014 again.

qiucx

thanks for your reply. Encoding::default_internal= "UTF-8" and Encoding::default_external= "UTF-8" are set in my entry main.rb and it works. the simple chinese character can be put correctly. However, when the split method can not work as expected. Following is the log and what is the problem with split method? thanks


def getUsersInfoFromFile(fileName, path)
        arrayAttributes = Array.new
        pathAttrFile = Sketchup.find_support_file fileName, path
        if (pathAttrFile)
            fileAttr = File.open pathAttrFile,"r;utf-8"
            arrayFileLines = fileAttr.readlines

            arrayFileLines.each do|fileline|
                arrayFileLine = fileline.split ","
                arrayAttributes << arrayFileLine
            end
            fileAttr.close
        end
        return arrayAttributes
    end

also, when arrayFileLines = fileAttr.readlines is changed to arrayFileLines = fileAttr.encode("UTF-8").readlines, the problem remains and split error can be fixed when fileAttr.encode("ISO-8859-1").readlinesis used. can you explain? thanks again.

Error:
#<ArgumentError: invalid byte sequence in UTF-8>
C:/Users/tc/AppData/Roaming/SketchUp/SketchUp 2014/SketchUp/Plugins/CRSD/RequireFiles/UserManager/UserManager.rb:130:in split' C:/Users/tc/AppData/Roaming/SketchUp/SketchUp 2014/SketchUp/Plugins/CRSD/RequireFiles/UserManager/UserManager.rb:130:in readLines'
C:/Users/tc/AppData/Roaming/SketchUp/SketchUp 2014/SketchUp/Plugins/CRSD/RequireFiles/UserManager/UserManager.rb:138:in getUsersInfoFromFile' C:/Users/tc/AppData/Roaming/SketchUp/SketchUp 2014/SketchUp/Plugins/CRSD/RequireFiles/UserManager/UserManager.rb:27:in checkLogin'
C:/Users/tc/AppData/Roaming/SketchUp/SketchUp 2014/SketchUp/Plugins/CRSD/RequireFiles/UserManager/UserManager.rb:97:in block in login' C:/Users/tc/AppData/Roaming/SketchUp/SketchUp 2014/SketchUp/Plugins/CRSD/RequireFiles/UserManager/UserManager.rb:106:in call'
C:/Users/tc/AppData/Roaming/SketchUp/SketchUp 2014/SketchUp/Plugins/CRSD/RequireFiles/UserManager/UserManager.rb:106:in show_modal' C:/Users/tc/AppData/Roaming/SketchUp/SketchUp 2014/SketchUp/Plugins/CRSD/RequireFiles/UserManager/UserManager.rb:106:in login'
C:/Users/tc/AppData/Roaming/SketchUp/SketchUp 2014/SketchUp/Plugins/CRSD/RequireFiles/Main.rb:7:in <top (required)>' C:/remove/remove_SketchUp/Tools/RubyStdLib/rubygems/core_ext/kernel_require.rb:45:in require'
C:/remove/remove_SketchUp/Tools/RubyStdLib/rubygems/core_ext/kernel_require.rb:45:in require' C:/Users/tc/AppData/Roaming/SketchUp/SketchUp 2014/SketchUp/Plugins/CRSD.rb:1:in <top (required)>'

qiucx

i find the solution but can not know the reason.

fileline.force_encoding("ISO-8859-1").encode("utf-8", replace: nil)

can anyone explain it? And does anyone have better solution?

thanks.

tt_su

Can you share your file? The error indicates you are opening a file in UTF-8 mode without the file actually being UTF-8. Are you sure it's UTF-8 encoded?

qiucx

the files attached.

userAttributes.csv

UserManager.rb

Login.zip

TIG

It's not the RB that is wrongly encoded - that is properly encoded as UTF8-without-BOM anyway.
BUT related files like the CSV [and HTML?] are also best when similarly encoded - but they are not.

By changing the v2014 read in string back and forth it makes it acceptable.
Note that the encoding change will break earlier SketchUp version users so you need to check if it's defined before using it.
Also writing/reading a file to the main folders could have issues with permissions with earlier versions <v2014 AND break if the user has a custom folder setup

qiucx

BUT related files like the CSV [and HTML?] are also best when similarly encoded - but they are not.

I have used the notepad++ to change the files encoding to UTF-8 without BOM.

"also best when similarly encoded - but they are not" how can i to do this similarly encoded?

TIG

Sorry, you misunderstand me.
'Similarly encoded' simply means encoded as UTF8-without-BOM - which you say you have now done ?

qiucx

I have set all files to UTF-8 without BOM.

but the error remain:

Error: #<ArgumentError: invalid byte sequence in UTF-8>
C:/Users/tc/AppData/Roaming/SketchUp/SketchUp 2014/SketchUp/Plugins/CRSD/RequireFiles/UserManager/UserManager.rb:24:in `split'

Can anyone have it? thanks.

TIG

Try this version...
I have marked changes with a final ###
Also try replacing the Chinese with some Western text in case of issues there ?

UserManager.rb

qiucx

Thanks for all. The problem is done.

The solution is to use ultraedit convert function(convert ascii to utf-8(unicode edition)). Originally, i use the notepad++ to convert utf-8 without BOM, and it seems not work as expected.

Now, the split function can work correctly without any force_encoding, but the input from html should be converted using force_encoding("UTF-8").

Thanks again for TIG.