Dear All,
I’m trying (and getting a very hard time) to use “targets” to improve an existing R-code.
I’m a newbie on targets… so you must bear some patience with me
The process heavily depends on a programming pattern that (apparently… or is just my lack of knowledge ) is not supported/handled by target.
As an example, suppose we have two files, each with his own read function. (Important: This is usually read from a configuration file during script run). Typically this will be a variable on global env
fd_read <- data.frame(
fnme = c("file1.csv", "file2.xlsx"),
rd_func = c("read.csv", "read.xlsx")
)
(Important2: The above functions are from packages, but typically they are custom.)
Now… (and its here the things go off rails!!) in targets pipeline I can define the files as a target
using (static or dynamic) branching and even execute the read.
fd_read <- data.frame(
fnme = c("file1.csv", "file2.xlsx"),
rd_func = c("read.csv", "read.xlsx")
)
list(
tar_files(
the_files,
fd_read$fname
),
tar_target(the_func,
command = as.character(fd_read[which(the_files == fd_read$fname), "rd_func"]),
pattern = map(the_files)
),
tar_target(data,
command = get(the_func)(fd = the_files),
pattern = map(the_func, the_files)
)
)
This seems to work fine… but it doesn´t.
Problem: If a function (e.g “read.csv”) is changed targets
does not re-run the pipeline. It seems that “getting” the functions by name makes them outside the tracking of targets.
I tried multiple variations to this (including creating a special environment for these functions…in which case targets
identify all functions as always changed), but in no situation I was able to get the correct result:
- If a file change: run the specific read o that file.
- If a read function changes: run the read of all files that use that function.
Thank you very much in advance for any help.
Best Regards.
A.Combo